The recent transformations by Fivetran, one of the leading firms in data integration, on product enhancement has increased the pace of data transactions. The new features will help in reducing the data complexities of modern data stacks, reducing costs through extract-load-transformation (ELT) automation and decisions based on data.
Data engineers survey 2021 found that most of the key data set used by companies won’t benefit in decision making and 68% of data engineers don’t have time for extracting min data value. The new upgrade will minimise the complexity of the data transformation process and provide companies with more reliable data sources, which will help in decision-making.
Users will be able to automatically run their dbt Core models by scheduling them following Fivetran connector sync completion. The new feature will reduce the data and accelerate the end-to-end ELT pipeline, which helps the users save money by providing already computed and updated data.
The pre-built model provided by Fivetran helps with faster analytics, and the packaged SQL scripts help in generating data quickly without a lot of data engineering overhead. Fivetran has provided new degrees of data automation by combining the data model with integrated scheduling, which focuses on an optimised approach to ELT.
“It’s very valuable to have Fivetran as one provider who can take care of both steps — getting data into the destination through its connectors, and then using dbt to transform the data in the data warehouse,” said Gustav Lindqvist, Chief Data Officer at Meditopia.
Users can better track and manage the data with the help of a data lineage graph and can also manage end-to-end pipelines by providing the facility of better observability features. The visual representation allows the data engineers to go through the data map, which will accelerate the transformation process.
“Fivetran’s ability to orchestrate dbt Core models brings the E, L, and T together and eliminates previous gaps in the process. Our users are clear about their need for the freshest data while controlling their transformation costs,” said Fraser Harris, Vice President of Product at Fivetran. “With Fivetran Transformations, our complete ELT data pipelines empower our customers to make revenue-impacting, data-driven decisions. This is fulfilling Fivetran’s mission to make access to data as simple and reliable as electricity.”
“Fivetran’s integrated scheduling is quite powerful. Given our customers consume data in the standard format, it’s very important for us to tightly couple the data sync with its transformation to avoid data being out of sync and to speed up the whole process,” said Albert Gozzi, CEO and co-founder at Aleph. “Having more control over when to run the transformations ensures fresh, timely data, and doing so within Fivetran removes the need to develop and maintain that functionality ourselves.”
