150k downloads: the dataset lab training AI to use computers

Most AI models can answer questions. Very few can open Photoshop, run a workflow in AutoCAD, and click the right button at the right moment. Markov, a Y Combinator-backed company, is building the ai training datasets that close that gap — 10,000+ hours of real computer-use recordings, synced mouse and keyboard events, and verified outputs across CAD software, browsers, design tools, and more.

Why Computer-use AI Needs a Different Kind of Data

Text and image datasets are everywhere. The bottleneck for the next generation of AI — agents that can actually operate software — is behavioural data: what does a human do, step by step, inside a real application?

This is the gap Markov was built to fill. The company's thesis, as stated in their YC launch, is that you cannot train a reliable computer-use agent on scraped screenshots or synthetic clicks. You need synchronized screen recordings, real mouse and keyboard events, narrated reasoning, and a verified correct output to check against. That combination is what makes a model learn how to use software, not just recognise what it looks like.

The market for this is real and growing fast. Frontier AI labs — the handful of companies building foundation models — need millions of hours of this kind of data. Markov's open dataset on HuggingFace already has 150k+ downloads, which is a strong signal that the demand is not theoretical.

For founders building products on top of AI tools, this matters: the quality of the underlying training data is what determines whether an AI agent can actually complete a task in your software stack, or just hallucinate its way through a workflow. Understanding how AI agents are being trained to operate business tools is becoming a prerequisite for evaluating any AI automation vendor.

What Markov Actually Captures — and Why the Detail Matters

Markov's dataset is not a collection of screenshots. Each recording is a complete workflow: a screen capture synchronized with every mouse movement and keystroke, a spoken narration from the person doing the task, a problem statement, and a verified gold-standard output file with a scoring rubric.

That last piece — the verified output — is what separates useful training data from noise. A model trained only on recordings learns to mimic actions. A model trained on recordings plus verified correct outcomes learns to solve problems. The rubric tells the model not just what was clicked, but whether the result was actually right.

The domains covered span CAD software (AutoCAD, SolidWorks, Siemens NX, SketchUp, Revit, CATIA), browsers, coding environments, design tools like Blender and Photoshop, spreadsheets, and enterprise platforms like Salesforce. That breadth is deliberate — a computer-use agent that only works in one application is not commercially useful to a frontier lab.

Markov also has 500+ hours of gaming data across Valorant, Minecraft, and GTA — which sounds like a side project but is actually a smart move. Gaming environments are high-frequency, high-precision input scenarios that stress-test a model's ability to track fast, complex human behaviour. It is a data diversity play, not a distraction.

CAD Environments: the Hardest Dataset Nobody Else Built

The CAD dataset is where Markov's moat is clearest. Their cad-environments release on HuggingFace contains 51 complete task workflows totalling 99 hours, across eight software categories covering mechanical design, architecture, and structural analysis. Their larger cad-1000-hours dataset runs to 1,018 hours across 509 tasks and 7 CAD applications.

Why is CAD hard? Because CAD software is used by specialists — mechanical engineers, architects, product designers — who take years to become proficient. Capturing expert workflows in AutoCAD or Siemens NX is not something you can crowdsource cheaply. You need to find people who actually know the software, record them doing real tasks, verify the outputs against engineering standards, and annotate the reasoning. That is expensive and slow, which is exactly why no one else has done it at this scale.

This is the same logic that makes any expert-knowledge dataset defensible: the barrier is not the technology, it is the human expertise required to generate the ground truth. A frontier AI lab trying to train a CAD agent cannot fake this data or generate it synthetically — they need real expert recordings, and right now Markov is the primary source.

For context on how niche, expert-domain data creates durable business value, the same pattern shows up in how early-stage startups build defensible positions by solving problems that require genuine domain depth.

150k Downloads: What the HuggingFace Traction Signals

150,000+ downloads on HuggingFace is not a vanity metric for a data company. HuggingFace is where AI researchers and lab engineers actually go to find training data — it is the distribution channel that matters for this market. A high download count on HuggingFace means frontier lab engineers are already pulling Markov's data into their pipelines.

The open-source computer-use dataset alone covers 12,300 hours of screen recordings across 48,478 videos, released under a CC-BY-4.0 licence. That is a meaningful research contribution, and it creates a feedback loop: researchers who use the open data cite it, labs that see it cited evaluate the paid premium datasets, and Markov's brand compounds in the only community that matters for their sales cycle.

The Echoverse product — deep, evolving environments for computer-use agents — extends this further. Rather than static recordings, Echoverse provides dynamic environments where agents can be tested and trained iteratively. That is a significant expansion of the product surface: from data to data plus evaluation infrastructure.

YC Backing and What It Means for the Market

Markov is a Y Combinator company. YC's public listing describes them as building "expert computer-use datasets for frontier AI labs" — which is a precise description of a very specific customer: the handful of organisations in the world training foundation models.

YC backing matters here for two reasons. First, it signals that the market is real — YC does not back data infrastructure companies unless there is clear evidence of enterprise demand. Second, it gives Markov access to a network of AI lab founders and researchers who are exactly the buyers they need to reach.

Co-founder and CEO Dev Mandal has been active on X sharing the dataset launches and research updates. The YC launch post framed Markov's core insight clearly: the next frontier in AI capability is not bigger models, it is better behavioural data. That framing is resonating — the Digg coverage of their CAD dataset announcement drew 45,700+ combined views.

Open-source as a Business Strategy, Not a Charity Move

Markov releases significant portions of their data openly on HuggingFace. This is not altruism — it is a calculated distribution strategy.

The logic: open-source data gets downloaded by researchers who publish papers citing it. Those papers get read by AI lab engineers who need production-quality training data at scale. Those engineers become paying customers for Markov's premium datasets — the ones with full annotation, verified outputs, rubrics, and CAD-specific expert workflows that are too expensive to release for free.

This is the same flywheel that open-source developer tools have used for years: give away the thing that builds trust and awareness, sell the thing that requires scale and quality guarantees. The 150k download number is the top of that funnel.

The risk is real — a well-resourced lab could use the open data to bootstrap their own collection operation. But Markov's moat is not the data format, it is the operational capability to recruit domain experts (CAD engineers, architects, Salesforce admins), instrument their workflows, verify outputs, and do it at scale across dozens of applications. That is hard to replicate quickly.

The Data Quality Bar That Separates Markov From Clip Scrapers

The computer-use data market has a quality problem. It is easy to scrape screen recordings from YouTube tutorials or game replay files. It is very hard to produce synchronized, annotated, expert-verified workflows with correct outputs.

Markov's datasets include problem statements (what the user was asked to do), input reference files, gold output files (the correct result), and scoring rubrics. That structure means a model trained on this data can be evaluated — you can check whether it produced the right output, not just whether it clicked in roughly the right place.

This is a meaningful technical distinction with a direct business implication: AI labs building products that need to reliably complete tasks (not just demo well) will pay a significant premium for data that includes verifiable outcomes. Demos that fail in production are a credibility and revenue problem — the same principle that applies when evaluating any AI tool for your business stack.

The gaming dataset (500+ hours of Valorant, Minecraft, GTA) is also higher quality than it sounds — synchronized mouse and keyboard inputs at gaming-level precision is a stress test for temporal alignment that most screen recording datasets don't attempt.

What This Means for Founders Building on AI Tools

If you are a founder evaluating AI agents or automation tools for your business, Markov's existence changes one question you should be asking vendors: what data was your model trained on?

An AI agent that claims to automate your Salesforce workflows, your CAD review process, or your design production pipeline is only as reliable as its training data. If that data was synthetic, scraped, or limited to simple click sequences, the agent will fail on real, complex workflows — the ones that actually matter for your business.

Markov is one of a small number of companies building the data infrastructure that makes reliable computer-use AI possible. Their focus on complete workflows with verified outputs is the right approach for production-grade agents. Founders who understand this will ask better questions before committing budget to AI automation tools.

The Business Model: Who Actually Pays for This Data

Markov's paying customers are frontier AI labs — the organisations training foundation models that need to develop computer-use capabilities. This is a small, high-value customer base: a handful of labs globally, each willing to pay significantly for high-quality, domain-specific training data.

The open-source datasets on HuggingFace serve as the top-of-funnel. The premium paid datasets — with full annotation, CAD expert workflows, verified outputs, and the Echoverse evaluation environments — are the product. The business model is essentially enterprise data licensing, with open-source as the distribution and trust-building layer.

This is a defensible position if Markov can maintain quality and expand domain coverage faster than labs can build internal data operations. The CAD dataset is the strongest example of this: 1,018 hours of expert CAD workflows is not something a lab can replicate in a quarter.

Conclusion

Markov is solving a specific, hard problem: capturing how experts actually use software and turning it into training data AI labs can use. Their 150k+ HuggingFace downloads, 10,000+ hours of recordings, and YC backing all point in the same direction. For founders evaluating AI agents, the lesson is simple — ask what the model was trained on before you commit. Data quality determines whether an AI agent works in production or just in a demo.