Data Exchange Tools and Utilities (DExT) Repositories and Preservation Tools QuDEx

Data conversion and proprietary data entry and analysis are particularly important and problematic aspects of data management and curation. The Data Exchange Tools and Conversion Utilities (DExT) project aimed to provide researchers and support staff working with primary research data with a suite of tools that will enable data to be long-term curated and exchangeable. Much important primary research data is created every day in the course of academic and policy research. While Data Sharing Policies are encouraging sharing and formalised archiving of data, the ideal life cycle for data creation to re-use remains beset by obstacles. The main issues involve the buying-in to a dedicated analytic strategy and typically a particular software package. Over the years the UKDA has seen a number of such softwares quickly become obsolete. To address the problem of incompatibility between software various data conversion tools have come of the market. One example for numeric and statistical data is StatTransfer and DBMSCopy that enable conversion from say SPSS to Stata. Equally the development of the format SPSS.por in the 80s enabled import and export between the major statistical analytic packages. However in the qualitative data analysis software field there are no such inter-software conversion tools. The rationale for the DExT work is that open data exchange formats are necessary for maximising the opportunities for data sharing and long-term archiving. The project led by the UK Data Archive researched, developed, refined and tested models for data exchange for both survey data and qualitative research data based on XML/RDF schema and developed some demonstrator tools for data import and export. The key objective was to develop an intermediate data exchange format that is as vendor neutral as possible. Software and platform neutral formats clearly have huge advantages for long-term archiving and for portability – passing between data analysis softwares. In order for take up of and sustainability of a format/standard, ideally they need support from software vendors to enable import and export functionality. The project worked closely with a number of key software vendors to loath these possibilities. The data formats included in the demonstrator project are those that are commonly used in research predominantly, but not exclusively in the social sciences – for survey data: SPSS, STATA, CSV and qualitative data: word, Atlas-ti, MaxQDA and Nvivo. These formats are also typically found across all domains of primary research. A small scale evaluation of the XML schema and data conversion tools was carried out in order to inform JISC of the most viable options for future development in this area. A longer-term aim of this work might be to build a fully functional and scalable facility or service where data formats can be submitted and seamlessly returned in a chosen, desired format – via a neutral exchange format– a true data exchange service. The project was divided into two separate and quite distinct work packages: The first worked with qualitative social interview-based multimedia data and produced and tested a formal XML schema to support data exchange. This work and the schema was termed QuDex (for qualitative data exchange). A version 3 of the QuDEx schema was released for public comment plus its accompanying documentation, and XML instance files. These are published on the DExT website and will be available on the Open Data Foundation (ODaF) website. A METS file was also produced. The second worked with survey data, namely in SPSS format and created a java-based data conversion tool ((Survey-DExT), which exported multiple version of SPSS to an ‘open’ format plus associated formal metadata, and to other major data analysis formats (STATA, SAS). The tool is built so that it is completely extensible to conversion from other data formats such as STATA and SAS and so on. The open-source conversion tool is fully documented and downloadable from the Open Data Foundation website. The work under the small scale DExT project has contributed to the Repositories Programme by investigating and developing a formalized XML schema and some open source demonstrator tools. These form as solid grounding on which to develop internationally agreed standards and tools that could be widely used to support ingest, long-term preservation, and dissemination of social science research data. This project was formally part of the Tools and Innovation strand, which aimed to develop and pilot innovative approaches to repository use and digital preservation through the development of new software and tools. The outputs are relevant to i) the Discovery to Delivery stream, as the proposed service is based upon common standards for data interoperability, and (ii) Shared Infrastructure Services for resource discovery, repositories and curation - machine to machine services that support rights, profiling, terminologies, registries, file format and representation etc. The project supported the Programme's desire to improve the efficiency and quality of repository functions, by helping automate the processes of data conversion, and by providing SMART data and tools - in the form of a universal data exchange format. The work set out has thus contributed to the need for refinement of the application of standards and specifications for digital repositories and preservation by building software and tools for both digital repository use and digital preservation. An immediate and indeed longer-term benefit should be increase in productivity for preservation and data sharing services and enhance both the reprocessing of legacy datasets and the data refreshment element vital to good data preservation practice.