Skip to product menu
close
  • Recent Launches
    Press Space or Enter to display list of options
EXPLORE ALL PRODUCTS

Recent Launches

New

Payroll software with automated tax payments and filing.

Try now
New

Robotic process automation software to automate high-volume, rule-based tasks.

Try for free
New

Low-code IoT platform and solutions for connected businesses.

Try now
New

Business formation service to launch and grow your businesses.

Try now
New

Privacy-friendly application analytics solution.

Try for free

Sales

 
CRM

Comprehensive CRM platform for customer-facing teams.

CRM
 
Bigin

Simple CRM for small businesses moving from spreadsheets.

Bigin
 
Forms

Build online forms for every business need.

Forms
 
SalesIQ

Live chat app to engage and convert website visitors.

SalesIQ
 
Bookings

Appointment scheduling app for consultations with customers.

Bookings
 
Sign

Digital signature app for businesses.

Sign
 
RouteIQ

Comprehensive sales map visualization and optimal route planning solution.

RouteIQ
 
Thrive

Complete loyalty and affiliate management platform.

Thrive
 
Voice

Cloud Contact Center Software for businesses.

Voice
 
Suites
CRM Plus

Unified platform to deliver top-notch customer experience.

CRM Plus

Marketing

 
Social

All-in-one social media management software.

Social
 
Campaigns

Create, send, and track targeted email campaigns that drive sales.

Campaigns
 
Forms

Build online forms for every business need.

Forms
 
Survey

Design surveys to reach and interact with your audience.

Survey
 
Sites

Online website builder with extensive customisation options.

Sites
 
PageSense

Website conversion optimization and personalisation platform.

PageSense
 
Backstage

End-to-end event management software.

Backstage
 
Webinar

Webinar platform for webcasting online webinars.

Webinar
 
Marketing Automation

All-in-one marketing automation software.

Marketing Automation
 
LandingPage

Smart landing page builder to increase conversion rates

LandingPage
 
Publish

Manage all your local business listings on a single platform.

Publish
 
SalesIQ

Live chat app to engage and convert website visitors.

SalesIQ
 
Sign

Digital signature app for businesses.

Sign
 
Thrive

Complete loyalty and affiliate management platform.

Thrive
 
Voice

Cloud Contact Center Software for businesses.

Voice
 
NEW
LeadChain

Sync, manage, and convert leads across channels seamlessly.

LeadChain
 
NEW
CommunitySpaces

Online community platform for individuals and businesses to grow their network and brand.

CommunitySpaces
 
Suites
Marketing Plus

Unified marketing platform for marketing teams.

Marketing Plus

Commerce and POS

 
Commerce

eCommerce platform to manage and market your online store.

Commerce

Service

 
Desk

Helpdesk software to deliver great customer support.

Desk
 
Assist

Remote support and unattended remote access software.

Assist
 
Lens

Interactive remote assistance software with augmented reality.

Lens
 
FSM

End-to-end field service management platform for service businesses.

FSM
 
SalesIQ

Live chat app to engage and convert website visitors.

SalesIQ
 
Voice

Cloud Contact Center Software for businesses.

Voice
 
NEW
Solo

The all-in-one toolkit for solopreneurs.

Solo
 
Bookings

Appointment scheduling app for consultations with customers.

Bookings
 
Suites
Service Plus

Unified platform for customer service and support teams.

Service Plus

Finance

 
Books

Powerful accounting platform for growing businesses.

Books
 
FREE
Invoice

100% Free invoicing solution.

Invoice
 
Expense

Effortless expense reporting platform.

Expense
 
Inventory

Powerful stock management and inventory control software.

Inventory
 
Billing

End-to-end billing solution for your business.

Billing
 
Checkout

Collect payments online with custom branded pages.

Checkout
 
NEW
Payroll

Payroll software with automated tax payments and filing.

Payroll
 
NEW
Solo

The all-in-one toolkit for solopreneurs.

Solo
 
Practice

Practice management software for accounting firms.

Practice
 
Sign

Digital signature app for businesses.

Sign
 
Commerce

eCommerce platform to manage and market your online store.

Commerce
 
Suites
Finance Plus

All-in-one suite to manage your operations and finances.

Finance Plus

Email and Collaboration

 
Mail

Secure email service for teams of all sizes.

Mail
 
Meeting

Online meeting software for all your video conferencing & webinar needs.

Meeting
 
Writer

Word processor for focused writing and discussions.

Writer
 
Sheet

Spreadsheet software for collaborative teams.

Sheet
 
Show

Create, edit, and share slides with a sleek presentation app.

Show
 
Notebook

Beautiful home for all your notes.

Notebook
 
Cliq

Stay in touch with teams no matter where you are.

Cliq
 
Connect

Employee experience platform to communicate, engage, and build positive employee relations.

Connect
 
Bookings

Appointment scheduling app for consultations with customers.

Bookings
 
TeamInbox

Shared inboxes for teams.

TeamInbox
 
WorkDrive

Online file management for teams.

WorkDrive
 
Sign

Digital signature app for businesses.

Sign
 
Office Suite

Powerful collaborative work platform for teams.

Office Suite
 
Office Integrator

Built in document editors for web apps.

Office Integrator
 
ZeptoMail

Secure and reliable transactional email sending service.

ZeptoMail
 
Calendar

Online business calendar to manage events and schedule appointments.

Calendar
 
Learn

Knowledge and learning management platform.

Learn
 
Voice

Cloud Contact Center Software for businesses.

Voice
 
ToDo

Collaborative task management for individuals and teams.

ToDo
 
Tables

Work management tool to connect people, processes, and information.

Tables
 
FREE
PDF Editor

Collaborative online PDF editing tool.

PDF Editor
 
Suites
Workplace

Application suite built to improve team productivity and collaboration.

Workplace

Human Resources

 
People

Organize, automate, and simplify your HR processes.

People
 
Recruit

Intuitive recruiting platform built to provide hiring solutions.

Recruit
 
Expense

Effortless expense reporting platform.

Expense
 
Workerly

Manage temporary staffing with an employee scheduling solution.

Workerly
 
NEW
Payroll

Payroll software with automated tax payments and filing.

Payroll
 
Shifts

Employee scheduling and time tracking app.

Shifts
 
Sign

Digital signature app for businesses.

Sign
 
Suites
People Plus

Comprehensive HR platform for seamless employee experiences.

People Plus

Security and IT Management

 
Creator

Build custom apps to simplify business processes.

Creator
 
Directory

Workforce identity and access management solution for cloud businesses.

Directory
 
FREE
OneAuth

Secure multi-factor authenticator (MFA) for all your online accounts.

OneAuth
 
Vault

Online password manager for teams.

Vault
 
Catalyst

Pro-code platform to build and deploy your apps.

Catalyst
 
Toolkit

Complete resource for any admin-related lookup queries.

Toolkit
 
Lens

Interactive remote assistance software with augmented reality.

Lens
 
Assist

Remote support and unattended remote access software.

Assist
 
QEngine

Test automation software to build, manage, execute, and report testcases.

QEngine
 
NEW
RPA

Automate manual, tedious, and repetitive tasks easily.

RPA

BI and Analytics

 
Analytics

Modern self-service BI and analytics platform.

Analytics
 
Embedded BI

Embedded analytics and white label BI solutions, tailored for your needs.

Embedded BI
 
DataPrep

AI-powered data preparation service for your data-driven organization.

DataPrep
 
NEW
IoT

Harnessing IoT analytics for real-time operational intelligence.

IoT

Project Management

 
Projects

Manage, track, and collaborate on projects with teams.

Projects
 
Sprints

Planning and tracking tool for scrum teams.

Sprints
 
BugTracker

Automatic bug tracking software for managing bugs.

BugTracker
 
NEW
Solo

The all-in-one toolkit for solopreneurs.

Solo

Developer Platforms

 
Creator

Build custom apps to simplify business processes.

Creator
 
Flow

Automate business workflows by creating smart integrations.

Flow
 
Catalyst

Pro-code platform to build and deploy your apps.

Catalyst
 
Office Integrator

Built in document editors for web apps.

Office Integrator
 
ZeptoMail

Secure and reliable transactional email sending service.

ZeptoMail
 
QEngine

Test automation software to build, manage, execute, and report testcases.

QEngine
 
Tables

Work management tool to connect people, processes, and information.

Tables
 
NEW
RPA

Automate manual, tedious, and repetitive tasks easily.

RPA
 
NEW
Apptics

Application analytics for all apps.

Apptics
 
Embedded BI

Embedded analytics and white label BI solutions, tailored for your needs.

Embedded BI
 
NEW
IoT

Build, deploy, and scale IoT solutions for connected businesses.

IoT
 
DataPrep

AI-powered data preparation service for your data-driven organization.

DataPrep

IoT

 
NEW
IoT

Low-code IoT platform and solutions for connected businesses.

IoT

Search Result

 
CRM Plus

Unified platform to deliver top-notch customer experience.

Try now
CRM Plus
 
Service Plus

Unified platform for customer service and support teams.

Try now
Service Plus
 
Finance Plus

All-in-one suite to manage your operations and finances.

Try now
Finance Plus
 
People Plus

Comprehensive HR platform for seamless employee experiences.

Try now
People Plus
 
Workplace

Application suite built to improve team productivity and collaboration.

Try now
Workplace
 
Marketing Plus

Unified marketing platform for marketing teams.

Try now
Marketing Plus
 
All-in-one suite

Zoho One

The Operating System for Business

Run your entire business on Zoho with our unified cloud software, designed to help you break down silos between departments and increase organizational efficiency.

TRY ZOHO ONE
Zoho One
Zoho Marketplace

With over 2000 ready-to-use extensions across 40+ categories, connect your favorite business tools with the Zoho products you already use.

EXPLORE MARKETPLACE
Marketplace

Zoho Analytics has been recognized in the 2024 Gartner® Magic Quadrant™ for ABI PlatformsRead more

Skip to main content

Data Pipeline

Organizations have large volumes of data from various sources, and the raw data needs to be prepared for analysis. The visual data pipeline builder in Zoho Analytics enables you to create data flows with multiple data stages and apply advanced data transformation functions to make the data ready for analysis. It significantly reduces time spent on data preparation by enabling you to automate the data preparation process.

Creating a Data Pipeline 

Creating a data pipeline comprises three major stages: Data Ingestion, Data Transformation, and Pipeline Execution.

Data Ingestion

Zoho Analytics allows you to import from various data sources, such as files and feeds, cloud storage, databases and data lakes, and business applications, to a workspace for analysis. Choose the tables and apply the necessary data cleansing and transformation functions to create a pipeline.

  1. Click the Create icon on the side navigation panel and choose Data Pipeline from the drop-down menu.
  2. From the table selection for the data pipeline pane, select the tables for creating a pipeline and click Next.
  3. The Pipeline builder powered by Zoho DataPrep window will open. 
  4. Stages are nodes created for processing data while applying data flow transforms. Every table selected will have a stage created by default.

Data Transformation

  1. Click the + icon and choose Prepare Data to open the DataPrep Studio page and explore more data transformations.
  2. Alternately, right click a stage and choose the necessary transformation function to be added to that stage.
    • Append: Combines data from two or more datasets by adding rows from one dataset to another, assuming both have the same structure (columns).
    • Join: Merges two datasets based on a common key, bringing together columns from both datasets.
    • Pivot: Converts unique values from one column into multiple columns.
    • Unpivot: Converts columns into rows, reversing a pivot operation, often used to normalize wide data into a long format.
  3. Once you are done creating a data flow, right-click the final stage and choose Create as an output table.

Pipeline Execution

Once you have created the output table, Zoho Analytics allows you to execute the data pipeline either through a Manual Run or by scheduling it for automatic execution. Any pipeline created is saved as a Draft initially. To add the pipeline as a table to the workspace, click the Draft option and change it to Mark as Ready. 

Manual run

Click the Run option in the top right corner. Each pipeline execution is saved as a job, and a summary is provided for each execution.

Scheduling Pipeline Execution

Scheduling pipeline automates the execution of a data pipeline at predefined intervals. Instead of running the pipeline manually each time, you can set up a schedule to trigger the pipeline automatically at specific times or frequencies.

To schedule a pipeline execution,

  1. Click the Schedule option in the top right corner.
  2. Choose the interval in which the pipeline should be executed from the Repeat option. The following are the supported intervals for execution.
    • Every N hours
    • Every Day
    • Weekly Once
    • Monthly Once
  3. Select the Time zone in which the pipeline execution should occur. By default, your local time zone will be selected.
  4. The Pause Schedule After option allows you to choose to stop the schedule after n number of failures.

Job Summary

A pipeline execution is called a Job; a job tracks the progress of imports, transformations, and exports in a pipeline. Each time a pipeline is executed, a Job summary is provided.

Overview

The Overview tab provides the general details about the pipeline execution as given below:

  • Pipeline Status: Indicates whether an execution was a success or a failure.
  • Duration: The time taken for execution.
  • Run by: The user profile who executed the pipeline.
  • Storage Used & Data Processed: Shows the total data storage and the number of rows used for executing the data pipeline.
  • Start & end time: Gives the time when the execution was started and completed.

Stages

The Stages tab provides a detailed summary of each stage in the pipeline, such as the tables used for creating the data flow, the transform functions applied over the tables, and the details of the final table to be exported.

Output

The Output tab displays the data quality, total number of rows consumed, data size, and status of the pipeline execution of the final table.

Edit Pipeline

The Edit Pipeline option allows you to modify the pipeline flow. Choosing this option will open the pipeline builder pane.


Job History 

Job History displays records of all pipeline executions, including details such as execution time, status, user who initiated the run, and the storage and rows consumed. Job with manual execution and sync schedule are also listed separately.

 

Manage Data Pipelines

The Data Sources tab lists all the connections configured and the data pipeline flows created within the workspace.
Click the Pipelines tab to view and manage the pipelines in the workspace. It provides general details about each pipeline status, such as Last Modified Time, Last Run Time, Next Schedule, and Schedule Status for each pipeline.

  1. Click any pipeline to access the Pipeline Builder to modify or execute the pipeline.
  2. Click the more icon to view the job history or to delete a pipeline
     
x
Thanks for your interest. We shall get back to you shortly.