How to Implement AI Across Fragmented and Legacy Systems

The short answer

Yes, you can implement AI across fragmented and legacy systems without replacing them first. The right approach depends on the workflow: how often it runs, how much data it touches, and how many systems it spans.

In the best case, your systems have official connectors to tools like Claude or ChatGPT, and you can work across them from a chat window with no integration build. More often, the data that matters most lives in an aging ERP or a niche industry platform. There, the answer ranges from a simple scheduled file export to a full data warehouse, with several practical options in between.

This guide walks through each option, when it fits, and the security mistakes to avoid along the way.

Why fragmented data is the norm, not the exception

Most middle-market companies, and especially PE-backed ones, run on a patchwork of systems. A typical portfolio company might have an ERP installed a decade ago, a CRM the sales team picked, a separate accounting package from an add-on acquisition, and spreadsheets filling the gaps between them.

Buy-and-build strategies make this worse. Each add-on brings its own stack, and integrating systems rarely makes the first 100-day plan. The result is that answering a simple question, like margin by customer across all locations, means pulling data from four places by hand.

The good news is that AI does not require a clean, unified data estate to start delivering value. It requires a deliberate choice about how data gets from each system to the AI, matched to the job you need done.

Best case: official MCP connectors

The easiest path exists when your systems offer official connectors to the AI tools your team already uses. Most of these connectors are built on the Model Context Protocol (MCP), an open standard that lets an AI assistant like Claude or ChatGPT read from, and sometimes act in, another application.

When every system in a workflow has an official connector, you can work across all of them from a single chat. For example, you could ask: "Pull the deals that closed last quarter from Salesforce, check the project status for each in monday.com, and summarize any delivery delays the account teams flagged in Slack." No integration project, no data pipeline, no developer. You connect each tool once, with your own login and permissions, and the assistant does the rest. AI platforms have automation tools so you can run this every quarter, day, or hour.

We typically see these connectors available for large, widely used platforms, because those vendors have the scale and incentive to build and maintain connectors. Salesforce, Slack, monday.com and Microsoft 365 are typical examples. If your workflow lives entirely in tools like these, start here before building anything. You can search for a connector in your preferred AI platform or the system you would like to connect to see if one is available.

The reality: your most important data probably has no connector

For most companies, and what we have seen first hand is the data that drives the business, such as orders, inventory, job costs, pricing and margins, usually lives somewhere less convenient.

That can mean an ERP that is ten or more years old, or one heavily customized for the business. It can also mean an industry-specific platform built for a single vertical, like construction, waste services, or distribution. Sometimes this can be an internal system the company built itself.

These systems rarely have official AI connectors, because their vendors are smaller or the software predates the trend. Some have a usable API, some only have a database underneath, and some only offer a reports screen.

A warning about unofficial MCP servers

When there is no official connector, it is tempting to search GitHub and install a community-built MCP server for your ERP or niche platform. Some of these are excellent. Large, well-maintained open-source projects with many contributors and active issue tracking. When company data is involved, we strongly recommend extreme caution and diligence.

The thing that makes MCP great is also what can make it dangerous in the wrong hands. An MCP server sits between an AI model and your system, holding credentials and executing actions on your behalf. A malicious or poorly built server could:

• Send your credentials or data to a third party without your knowledge.

• Give an attacker standing, unmonitored access to a core system like your ERP or file storage.

• Let a third party write, overwrite or delete records, not just read them.

• Be more susceptible to manipulation by hidden instructions planted in documents or web pages the AI reads, a technique known as prompt injection.

Before connecting any MCP server that the vendor did not publish, run through a basic diligence checklist:

• Who publishes it? Prefer software vendors, then established organizations, then widely adopted open-source projects. Avoid anonymous repositories with little history.

• Is it maintained? Look for recent commits, responsive maintainers and a real user base.

• Can it run read-only? Start with read access only. Grant write or delete permissions only when a workflow truly requires them.

• Whose credentials does it use? Best practices is to create a dedicated service account with the narrowest permissions possible, avoid any admin permissions.

• Has someone read the code? For anything touching sensitive systems, it's always best to have a developer review it or at least a LLM.

• Can you see what it does? Make sure actions are logged, and always test first,

When in doubt, building a small, purpose-built connector you control is often safer than adopting one you cannot vouch for. Many times public well maintained MCP servers do not exist for niche systems we are looking to connect.

Match the approach to the workflow

There is no single “best” architecture. The right choice balances four main factors: how often the workflow runs, how much data it touches, how many systems it combines, and how fresh the data needs to be. Most companies end up using several of these options at once, one per workflow.

1. Manual export to an AI tool

The simplest option is often the right first step. Export a CSV or Excel file from the system and upload it to Claude or ChatGPT. It costs nothing to try, proves whether the analysis is valuable, and tells you exactly which fields matter before you build anything.

2. Automated file exports

If the same export happens every week, automate it. Many older systems can email a scheduled report or drop a file on a server. Many AI platform have built in automation logic already. An automation, or an AI agent, can collect that file and run the same analysis each time. Flat files are unglamorous, but they are reliable and easy to audit.

3. Workflow automation platforms

Tools like Power Automate, Zapier, Make and n8n connect systems through their APIs, with prebuilt connectors for thousands of applications. They suit event-driven workflows that move a record from one system to another and add an AI step in the middle, like classifying, summarizing or drafting.

4. A custom connector or MCP server

When a team needs to ask a core system questions all day. If the system has an API or a database, a developer can build a small MCP server that exposes only the data and actions you approve. Because you build it, you control its security, which addresses the risks of unofficial servers described above.

5. A central data warehouse

When many systems feed the same questions, and we need one source of truth, it is time for a central data layer. Data from each source is loaded on a schedule into a warehouse such as Microsoft Fabric, Snowflake or Databricks, then cleaned and modeled into consistent definitions, like one meaning of "customer" or "gross margin" across every source.

This is the largest investment, but it is also what makes AI dependable at scale across complex environments. Dashboards, AI agents and LP reporting all draw on the same trusted numbers, and adding a newly acquired company becomes a repeatable process rather than a new project. Many firms grow into this option, starting with exports and adding a warehouse once the value is proven.

What if the system has no API at all?

We have come across many systems that even have neither an API or connector. They are still reachable, roughly in this order of preference:

• Most on-premise ERPs store data in SQL Server, Oracle or a similar database. A read-only replica or reporting copy can feed a warehouse or a custom connector without touching the live system.

• If the system can email or save a report on a schedule, that file becomes the integration.

• Ask the vendor. Many offer bulk export, data feeds or a hosted reporting database, sometimes for an added fee.

• As a last resort, software can log in and download data the way a person would. It works, but can require a lot of maintenance depending on how often the page changes.

Questions to answer before you build

If you’re still unsure, here are six questions we recommend asking:

1. What decision or task is this workflow meant to improve, and who does it today?

2. How often does it need to run: once, weekly, daily, or continuously?

3. Which systems hold the data, and does each one have an official connector, an API, a database, or only reports?

4. Does the AI only need to read data, or does it need to write back or take action?

5. How fresh does the data need to be: last month, yesterday, or right now?

6. Will this workflow be repeated across other teams, sites or portfolio companies?

If the answers are "once" and "read-only," export a file and start today. If they are "continuously," "many systems" and "across the portfolio or organization," plan for a data warehouse, and use exports in the meantime to prove the value.

Frequently asked questions

Can you implement AI on top of legacy systems without replacing them?

Yes. Most AI workflows can draw data from legacy systems through file exports, databases, APIs or custom connectors. Replacing an ERP is rarely a prerequisite, and it is usually far more expensive than connecting to the one you have.

How do you build an AI workflow across multiple disconnected systems?

First, check whether each system has an official connector for your AI tool. For those that do not, choose the lightest option that fits how often the workflow runs: file exports for occasional work, automation platforms for event-driven tasks, custom connectors for frequent use, and a data warehouse when many systems feed the same reporting.

Do you need a data warehouse before using AI?

No. A warehouse becomes worth it when several systems feed the same questions, data volumes are large, or results must be consistent across companies. Many teams start with exports, prove the value, then invest in a warehouse.

Are third-party MCP servers safe to use?

Some are, but they deserve careful diligence. An MCP server holds credentials and can act inside your systems. Prefer official servers from the software vendor, start with read-only access, use a dedicated low-privilege account, and review the code of anything unofficial before connecting it to sensitive data.

How long does it take to connect AI to a legacy ERP?

It ranges from minutes for a manual export to a few weeks for a secure custom connector. A multi-system data warehouse typically takes a few months, depending on the number of sources and how clean the data is.

About BaseForge Advisors

BaseForge Advisors is an AI and data consultancy for middle-market private equity firms and their portfolio companies. We help teams connect AI to the systems they already run, from a first secure connector to a portfolio-wide data warehouse, and we start with the workflow that pays back fastest.

If you need help working through these questions across your portfolio, we’re the team for you.