Processing open source data with exchangeable components

What is this action about?

Many authorities need to process open source data from the internet and other sources, such as news articles, mapping data, open government data, etc. for purposes of their mission. The data needs to be processed by passing it through a number of software components (for example a component to do automatic language translation) before a human can effectively use it. For various processing steps software tools (commercial and non-commercial) already exist and are in use by authorities of EU Member States and EU institutions. However, it proves difficult to exchange and reuse tools between authorities since often they were developed only with the interoperability within an individual authority or country in mind. This leads to costly duplication of effort and hampers collaboration between authorities across different countries. To tackle this issue a set of best practices needs to be establised which defines minimum interoperability standards (based on previous standardisation and industry standards). A component that adheres to these standards can then more easily be reused for different tasks by multiple authorities across Europe. 

What are the objectives?

The objective is to define a set of best practice standards to make software components in the field of open source data processing exchangeable which allows cost-saving and less duplicated effort.

The best practice standards comprise mainly of: 

• Standards for Meta Data describing the processed data
• Guidelines on how to develop new and adapt existing components for better interoperability

Where possible, existing standards and industry best practices will be taken into account.

Furthermore, the project aims to develop a set of interchangeable components and a composited demonstrator application.

What are the benefits?

For citizens

• Faster service from authorities which are equipped with the latest technology

For EU institutions and other EU bodies

• Cost-savings because open source data processing components can be reused across institutions
• Faster reaction to changing requirements, since software can be reused and needs not to be developed from scratch

For Member States

• Cost-savings because software components from other EU Member States can be adopted
• Easier collaboration across authorities and countries since a best-practice approach is adopted 
• Adoption of a proven best-practice approach for developing new software components

What has been achieved?

• Development of the Registry of available processing components
• Development of the Existing Metadata Standards
• Development of the demonstrator application

What are the next steps?

Based on the outcome of the previous deliverables, the team has identified a suitable solution implementing the architectural principles agreed with stakeholders, which is the KNIME platform. The team is currently developing the Pilot composite application using core components for open source information processing.