Empowering data analytics with our data lake management software platform
Pipeline Data Ingest and Design with UI-guided Feed Creation
Create feeds into the data lake with a very easy and intuitive GUI. For example, locate a CSV file and bring it into the data lake via the generation of a Hive table. Kylo™ will generate a schema based on data brought into Hadoop.
Kylo™ uses Apache NiFi for orchestrating data pipelines. Adding a file kicks off a workflow in NiFi, and the power of NiFi can then be used to visualize those workflows (Figure 1)
(Figure 1 – Pipeline Data Ingest & Design with Kylo™’s easy to use UI)
Self Service and Data
Use Kylo™ to create data standardization and validation rules and policies for data brought into Hadoop. For example, use Kylo™ to mask credit card data which will preserve the last four digits for any data brought in.
Also, perform 100+built-in transformations using Kylo™’s Wrangler module. This essentially provides a user-friendly way to do complex Spark-based transformations (Figure 2)
Figure 2 – Kylo™ is data analyst-friendly with self-service capabilities including a visual SQL builder)
Data Profile and Discovery
Kylo™ provides metadata tracking including lineage, allowing data stewards and data scientists to quickly catalog, discover and qualify data. Or profile data with Kylo™ to perform data quality checks (Figure 3) and gain confidence in various data sets.
(Figure 3 – Sample and profile data with Kylo™. Leverage Google-like searching capabilities)
Gauge the health of various data feeds via dashboards and use Kylo™ to drill into outstanding issues such as data integration jobs not completing (see Figure 4). Also operations manager is valuable for SLA policy monitoring and enforcement.
With Kylo™, business and IT users can bring new data sources into their data lake, perform standard transformations and publish to target systems. In addition, Kylo™ tracks and records lineage (provides traceability for data objects) as they move through and are transformed within the data lake.
Because data orchestration and management is such a critical need, companies are seeking solutions from the commercial and/or open source marketplace. For companies that prefer the open source route, Kylo™ is an impressive solution with support from Think Big, one of the premier big data firms in the market.
Think Big can help turn your data into an asset for enterprise use. Kylo™ is in private beta now, so use these links below to view a demonstration or request beta access.