Home›Data›DataOps

XcellOps|DataOps

Data you can trust by 9 a.m. — tested, observed, governed and kept in India.

DataOps applies software engineering discipline to data: pipelines as code, automated tests and CI/CD, orchestration, observability and data contracts — with DPDP-ready lineage and retention. Run by XcellHost engineers and our 24×7 NOC.

Pipelines as code & CI/CD
Tests & data quality
Orchestration
Freshness, volume, schema
Data contracts
Lineage, DPDP & retention
India regions by defaultMumbai Tier IV or India cloud
Run on schedule06:00 IST · 7 tasks
Pipeline board · orchestrate RUNNING
01Orchestrated run · 7 tasks Sample data
Tasks done0/7
Rows loaded0
Freshness—
Powered byXcellHost DataOps
ISO 27001 · 27017 · 27018SOC 2 Type I & IIMeitY empanelledTier IV DC · Mumbai24×7 NOC + SOCSince 1999
Overview

Most data problems are found by the person reading the report.

A finance dashboard shows yesterday's numbers. A churn model trains on a half-loaded table. A chatbot answers from a policy that was withdrawn last month. None of these are infrastructure outages — the servers are fine. The data is late, wrong, incomplete or not allowed to be there. That is data downtime.

DataOps is the discipline that prevents it. Pipelines are written as code and reviewed. Every change runs through automated tests. Orchestration runs jobs in the right order and retries the safe ones. Monitors watch freshness, volume and schema. Contracts stop producers from breaking consumers. Lineage shows where each column came from and where personal data flows — which is exactly what the DPDP Act will ask you to show.

Latethe table did not refresh by its deadline
Incompletehalf the rows arrived, or none
Wrongnulls, duplicates, impossible values
Brokena column renamed upstream, silently
Not allowedpersonal data kept past its purpose

Five kinds of data downtime. DataOps puts a test, a monitor or a rule in front of each.

DataOps principles

Eight rules we build every pipeline by.

Our working version of the ideas in the DataOps Manifesto (18 principles) and modern data engineering practice — written for teams that have to answer to auditors as well as analysts.

  1. Pipelines are codeSQL, Python and configuration live in Git, with pull requests, review and history. No hand-edited jobs on a server.
  2. Every change is testedUnit tests for logic, data tests for results — not null, unique, accepted values, row counts, referential checks.
  3. CI/CD for dataChanges build in an isolated environment on sample or cloned data, then promote to production automatically when green.
  4. Orchestrate, don't cronDependencies, retries, backfills and SLAs declared in an orchestrator, so a late source never feeds a half-built table.
  5. Observe the data, not just the jobA job can succeed and still load nothing. Monitor freshness, volume, schema and distribution per table.
  6. Agree contracts at the boundaryProducers publish a schema and quality promise; breaking changes are caught in CI before consumers feel them.
  7. Know your lineageEvery column traceable from source to dashboard, model or RAG index — for impact analysis and for privacy.
  8. Privacy by defaultPersonal data tagged at ingestion, masked where it isn't needed, kept in India and erased when its purpose ends.
DataOps lifecycle

Build on the left. Run on the right. Learn in the middle.

The DataOps loop has a development half and an operations half. Click a stage — or watch the run travel round.

BUILDRUN
01PlanBuild

—

    Capabilities

    What XcellHost DataOps covers.

    From the first Git repository to the AI index — run as one service.

    Pipelines as code & CI/CD for data

    Repository structure, branching and review rules; isolated build environments on cloned or sampled data; automatic promotion from dev to staging to production when tests pass. Rollback is a revert, not a midnight restore.

    lintunitdata testscontractdeploy

    Testing & data quality

    Schema, null, uniqueness, range, referential and freshness tests on every model, with severity levels that warn or block.

    Orchestration

    DAGs with dependencies, retries, backfills, SLAs and alerts — on a managed orchestrator such as Airflow* or Dagster*.

    Data observability

    Freshness, volume, schema, distribution and lineage monitored per table, with anomalies routed to the right owner.

    Data contracts

    Versioned schema and quality promises between producing and consuming teams, enforced in CI.

    Catalogue & governance

    Glossary, owners, PII tags, access policies and column-level lineage in a catalogue your teams actually use.

    DataOps for AI & RAG pipelines

    Documents and tables flow through the same discipline before they reach a model: source versioning, chunking and embedding jobs under orchestration, index freshness monitors, retrieval evaluation in CI and removal of withdrawn or personal content from the index. Pairs with RAG as a Service, Enterprise RAG and the managed vector database.

    Privacy & data security

    PII discovery, masking and retention jobs; posture from DSPM and leakage control with Purview DLP.

    Data observability

    Five signals that tell you the data is fit to use.

    Pick a signal to see what we watch, a sample monitor, and what happens when it fires.

    —

    —

    —
    When it fires—
    Data contracts

    A promise between teams, checked by a machine.

    The orders team owns the orders table; finance, the CRM and the AI assistant depend on it. The contract states the columns, types, owners, quality rules and personal-data tags. Try a change and see what CI says.

    contracts/orders.yamlv1.4
    
            
    ✓ CI passed—
    India context

    Purpose, retention, residency — written into the pipeline.

    Indian rules turn data engineering choices into legal ones. DataOps makes them repeatable and provable.

    11 Aug 2023DPDP Act, 2023 enacted
    13 Nov 2025DPDP Rules, 2025 notified; Board provisions in force
    13 Nov 2026Consent Manager provisions apply
    13 May 2027Notice, consent, security, breach, retention and rights duties apply
    Purpose limitation

    Tag every column with why it exists

    Under the DPDP Act personal data is processed for a specified purpose, and Section 8(7) requires erasure when consent is withdrawn or the purpose is no longer served — unless another law needs it kept. We carry purpose and legal-basis tags in the catalogue and lineage, so a withdrawal reaches every downstream table and index.

    Retention & erasure

    Retention as scheduled, tested jobs

    Rule 8 sets three-year inactivity erasure for large e-commerce, online gaming and social media platforms, with at least 48 hours' notice before erasure, and Rule 8(3) asks for relevant personal data, traffic data and logs to be kept for at least one year for specified purposes. We implement both as orchestrated jobs with evidence.

    RBI data localisation

    Payment data stays in India

    RBI's circular of 6 April 2018 requires payment-system data to be stored only in India. Its FAQ allows processing abroad, but the data must be deleted there and brought back to India within one business day or 24 hours of processing, whichever is earlier. We pin storage and pipelines to Indian regions.

    Cross-border transfer

    Know where every copy goes

    Section 16 lets the Government restrict transfers of personal data to notified countries, and stricter sector rules (RBI, SEBI, IRDAI) still apply. Lineage shows every cross-border copy — including SaaS tools and AI APIs — before a regulator or customer asks.

    Summaries of the DPDP Act 2023, DPDP Rules 2025 (as notified 13 November 2025) and RBI circular DPSS.CO.OD.No 2785/06.08.005/2017-18 and its FAQ, read November 2026. Orientation only, not legal advice; your organisation remains responsible for compliance. For consent, rights and breach workflows see the DPDP Compliance Platform.

    DataOps maturity model

    From heroics to habits.

    Five levels we use in assessments. The check below places you on it in two minutes.

    L1Ad hocScripts on a server, fixed by the one person who wrote them.
    L2RepeatableCode in Git and a scheduler, but tests and alerts are rare.
    L3DefinedOrchestrated, tested pipelines with CI and basic monitors.
    L4MeasuredObservability, contracts and lineage; data SLAs reported.
    L5OptimisedSelf-serve, policy-driven, privacy and cost automated.
    Tool · data-pipeline maturity assessment
      Your result
      0/ 100

      Answer the eight questions

      Unanswered questions count as "Not yet".

      Your 90-day roadmap
        Suggested package · —
        Tool · data-downtime cost estimator

        What does bad data cost you a month?

        Data downtime = incidents × (hours to detect + hours to fix). Put in your own numbers; the improvement percentages are assumptions you control, not promises.

        Assumptions with DataOps
        Illustrative · per month
        Data downtime today0 h
        With DataOps (assumed)0 h
        Cost today₹0
        Cost after₹0
        Avoided cost₹0
        vs DataOps Essentials ₹49,999*—
        Formula

        Downtime hours = N × (detect + fix). Cost = N × fix × engineers × engineer rate + downtime hours × business cost per hour. After = same formula with N × (1 − a), detect × (1 − b) and fix × (1 − c). All inputs are yours; no industry averages are assumed. Not a quote.

        DataOps for AI & RAG

        An AI assistant is only as current as its last pipeline run.

        Retrieval-augmented generation is a data pipeline with a model at the end. We run it like one.

        Sourceversioned documents and tables, owner and purpose tagged
        Clean & maskPII detected and masked, withdrawn content removed
        Chunk & embedorchestrated jobs, model version recorded
        Indexvector index freshness and size monitored
        Evaluateretrieval and answer checks in CI before release
        Serveanswers cite sources; feedback feeds the backlog
        How XcellHost delivers

        Data engineers by day. A NOC all night.

        DataOps needs both: engineers who write and review pipelines, and an operations desk that notices when the 3 a.m. run fails.

        Build team

        Data engineering

        • Pipeline and model development in your Git
        • Tests, contracts and CI set up per domain
        • Weekly change review with your owners
        Run team

        24×7 NOC & SOC, Mumbai

        • Failed runs and SLA misses picked up around the clock
        • Safe retries and backfills by runbook
        • Security alerts on data platforms triaged by the SOC
        Where it runs

        India by default

        • XcellHost Tier IV data centre in Mumbai
        • Or AWS, Azure, Google Cloud India regions
        • Storage and processing pinned to India
        Assurance

        Certified processes

        • ISO 27001, 27017, 27018 and ISO 20000-1
        • SOC 2 Type I & II · MeitY empanelled
        • Monthly data-SLA report and quarterly review
        Toolchain integrations

        Open standards first. Your platform second. Lock-in never.

        We operate the tools you have, and recommend open formats and orchestrators where you are starting fresh.

        OrchestrationApache Airflow*Dagster*Prefect*Cloud-native schedulers*
        Transformationdbt*Apache Spark*SQL & PythonNotebook jobs*
        Quality & testsGreat Expectations*Soda*dbt tests*Custom SQL checks
        Catalogue & lineageOpenLineageDataHub*OpenMetadata*Microsoft Purview*
        Storage & formatsApache Iceberg*Delta Lake*ParquetXcellHost S3-compatible storage
        Streaming & CDCApache Kafka*Debezium*Cloud streaming services*
        CI/CDGitHub Actions*GitLab CI*Azure DevOps*
        PlatformsAWS*Azure*Google Cloud*Databricks*Snowflake*

        * Product names are trademarks of their owners, used only to describe compatibility. No partnership or endorsement is implied unless stated on the vendor's own XcellHost page.

        Packages

        Priced by the pipelines we run — in rupees.

        Tier prices match Managed Data Lake Services, where the same team runs the platform. Cloud storage and compute are billed separately at actuals.

        Essentialsone platform, core pipelines
        ₹49,999/ month*
        • 15 pipelines, batch + CDC
        • Up to 10 TB managed
        • Daily quality checks
        • 8×5 support, 24×7 critical alerts
        • Monthly health report

        Build & onboarding ₹1,99,999* one-time

        Professionalstreaming, governance, 24×7Recommended
        ₹1,24,999/ month*
        • 60 pipelines incl. streaming
        • Up to 100 TB managed
        • Catalogue, lineage & masking ops
        • 24×7 NOC monitoring
        • Monthly FinOps review included

        Build & onboarding ₹4,99,999* one-time

        Enterprisemulti-cloud, dedicated pod
        On quoteafter assessment
        • 100 TB+ across clouds
        • Named data-engineering pod
        • Feature stores & RAG pipelines
        • Custom SLAs & audits
        • Quarterly architecture review

        Start with the ₹74,999* readiness assessment

        * Indicative — confirm with XcellHost at proposal. INR, excl. 18% GST. All prices are reused from existing XcellHost pages: tiers, build fees, extra pipelines (₹2,999*/month), readiness assessment and governance setup from Managed Data Lake Services; DSPM, Vector DB, RAG, XcellPrivacy and Database Operations from their own pages.

        Configure

        Size your DataOps service. See the first invoice.

        Lines go to the cart at price × quantity; GST is added at 18%. Enterprise comes as a quotation.

        Tier
        60 pipelines
        One-time Monthly add-ons
        XcellHost · estimateDataOps Professional
          Monthly₹0
          One-time₹0
          Subtotal₹0
          GST 18%₹0
          First invoice₹0

          * Indicative — confirm with XcellHost. INR. Cloud storage, compute and licences billed separately at actuals.

          FAQ

          DataOps, answered.

          DataOps is a way of building and running data pipelines with software engineering discipline: code in version control, automated tests, CI/CD, orchestration, monitoring of the data itself, and clear ownership. The aim is data that arrives on time, is correct, and is used only as allowed.

          DevOps ships application code. DataOps ships code and data together, and data changes every day even when the code doesn't. So DataOps adds data tests, freshness and volume monitors, contracts between teams and lineage on top of the usual CI/CD.

          Monitoring the data, not only the jobs: freshness (did it update on time), volume (did the expected rows arrive), schema (did columns change), distribution (are values in normal ranges) and lineage (what is upstream and downstream). A job can succeed and still load bad data; observability catches that.

          A versioned agreement, usually a YAML file, between the team that produces a dataset and the teams that use it. It lists columns, types, owners, quality rules and personal-data tags. CI checks every change against it, so a dropped or retyped column is caught before it breaks a report or model.

          You need an orchestrator; which one depends on what you already run. We operate leading open-source orchestrators such as Airflow* and Dagster* and cloud-native schedulers. For a fresh start we usually recommend an open-source option to avoid lock-in.

          Purpose and personal-data tags in the catalogue, lineage to every copy, masking where data isn't needed and retention as scheduled, tested erasure jobs. That makes Section 8(7) erasure and the Rule 8 retention periods provable. The DPDP Rules' core duties apply from 13 May 2027; compliance stays with you.

          Under RBI's April 2018 circular, payment-system data must be stored only in India. If it is processed abroad it must be deleted there and brought back within one business day or 24 hours. We pin storage and pipelines to Indian regions and show this in lineage.

          Yes. Document ingestion, masking, chunking, embedding and index refresh run as orchestrated, monitored pipelines, with retrieval evaluation in CI and withdrawn or personal content removed from the index. It pairs with our RAG, Enterprise RAG and vector database services.

          Essentials ₹49,999* per month (15 pipelines) and Professional ₹1,24,999* per month (60 pipelines), with build and onboarding of ₹1,99,999* or ₹4,99,999* one-time and ₹2,999* per extra pipeline per month. Enterprise is on quote. Indicative, excl. 18% GST; cloud usage at actuals.

          You do. Pipelines live in your Git repositories, data in your accounts or your XcellHost project. On exit we hand over repositories, runbooks, contracts and catalogue exports and remove our access.

          Tell us which report was wrong last month. We'll show you the pipeline fix.

          A free 30-minute DataOps review: your current pipelines, the three biggest downtime risks and a package price in rupees within two business days.*

          See packages → WhatsApp
          DATAOPS RUN
          BY XCELLHOST
          ISO 27001 · 27017 · 27018ISO 20000-1SOC 2 Type I & IITier IV · MumbaiSince 1999