At the moment, part of my work involves data integration from various sources, formats and locations into one major central database where important decisions are made from all the data ‘mined’.
I’ve evaluated a couple of products, from large enterprise scale level products(which does come with a big price tag) and down to simple macro/automation tools to perform ad hoc tasks(simple functionality and data reading like CSV files) on desktops and servers which can be used by normal non-programmer people.
One product I particularly found very powerful and useful is Talend Open Studio, which is an open source product but does come with company backing support just like MySQL does with their products as well. It is abit more advanced for a normal user, but programmers will find it quite a breeze.
With a powerful GUI with nice workflow modelling makes this very powerful, click and drag the connections and what actions you want to perform into place and welll….Bob’s your uncle. With a PERL/JAVA(you decide) backing you can build quite powerful data integration procedures and processes. I haven’t checked how, but I’m assuming as well that the extensibility could also be very powerful.