Integrate, process, analyze and publish data.
The Integration Hub is an online platform for:
- ETL processing, data integration and automation.
- Connecting to data stored in databases, text files and API's.
- Statistical, financial, analytical and machine learning operations.
- Sharing, publishing and visualizing data.
Why use the Integration Hub?
Integration with virtually any data provider. Built in support for all popular Sql databases, NoSql databases, file types, and third party API's.
You decide how and where to store your data. Whether secured on internal networks or published to the cloud, the integration hub will ensure you can keep your data safe.
Over 400 built in functions covering machine learning, finance, statistics and geometry. Plugin support available for more specific needs.
Publish & Share
Easily publish and share your data, charts and dashboards with other users of the platform.
Scale with Ease
Remote agents can be switch on/off on demand and can manage huge data volumes without impacting the performance for other jobs.
Data can be merged, transformed, cleaned, and profiled in many ways. Data can populate a data warehouse, integrate systems, or be published to users.
Features at a glance
- Single central access point
- Physical data stored and accessed anywhere
- Easy to install and scale
- Postgres (including Amazon Aurora)
- Microsoft Sql Server
- IBM DB2
- Azure Storage Tables
- Microsoft Excel
- Delimited Files (such as CSV)
- Json Files
- Xml Files
- REST based API's
- Local File Systems
- Azure Storage
- Bulk Load (for speed)
- Transactional Load (for data integrity)
- Source or Target based change detection
- Preserve change history (i.e. type 2 slowly changing)
- Sql like transforms (map, filter, group, join, union)
- Analytical aggregations (such as moving average)
- Join flat tables to structured datasets (such as JSON)
- Series transform to aggregate and even out time series.
- Row generation transform to pivot data.
- Hundreds of built in functions
- Machine Learning - Regression, Sentiment, Recommendation, Clustering, Anomaly detection
- Statistical - Curve fitting, Matrix, Distribution
- Financial - Interest rate, Yield, Discount etc.
- Live data - Weather, Stocks, Currency
- Trigger based on schedule (once off or recurring)
- Trigger based on file event
- Sequence processing based on completion of other jobs
- Custom data views
- Chart data (bar, pie, line, scatter etc.)
- User configurable dashboards which group views and charts into layout
- Publish tables/views/dashboard for private/public groups
- Create REST/JSON Api for external processes
- Strong encryption on passwords and marked sensitive fields.
- Encryption keys decentralized (i.e. cannot be accesses by Data Experts staff)
- Automatic SSL key generation to encrypt all data in transit
- Local network leakage prevention (ensures data never leaves local network).
- IP address whitelisting for remote processing agents access.
- Validate with any built in function
- Create custom/reusable data validations
- Validate via white lists or black lists
- Validate against lookups (i.e. value exists in other table)
- Invalid data can be rejected or stop processing
- Built in profiling functions
- Data type profiling
- Pattern profiling
- Capture profile data with any data processing
- Completely open source
- Data connectivity plugin (through Microsoft .Net Core)
- Function plugins (through Microsoft .Net Core)