An AI vendor demo is designed to show the smoothest path. A dealership evaluation has to inspect the messy path: stale inventory, duplicate customers, weak permissions, failed write-back, opt-outs, manager overrides, employee turnover, and contract exit.
The right vendor is not necessarily the company with the most features. It is the company that fits the store’s systems, exposes how the workflow operates, supports human control, and can be measured against a real baseline.
Score six areas, not the presentation
Use a 100-point scorecard. Require written answers and supporting documentation for high-risk claims.
| Area | Weight |
|---|---|
| Business fit and workflow depth | 20 |
| Integration and data quality | 20 |
| Security, privacy, and governance | 20 |
| Customer experience and human control | 15 |
| Measurement and reporting | 15 |
| Commercial terms and exit | 10 |
Business fit and workflow depth: 20 points
- Which exact dealership process does the product improve?
- What must the store change before launch?
- Which cases require human intervention?
- What does implementation look like after the kickoff call?
Ask the vendor to demonstrate your scenario, not its favorite one. Supply an anonymized difficult case and watch how the system handles missing data, ambiguity, and escalation.
Integration and data quality: 20 points
- Which CRM, DMS, scheduler, phone, inventory, and website systems are integrated today?
- Is the connection real-time, scheduled batch, manual export, or screen automation?
- What can the product write back, and where does the activity appear?
- How are duplicates, stale records, and conflicting fields handled?
“We integrate with your CRM” is not enough. Ask for the specific objects, fields, permissions, update frequency, failure alerts, and reference customers using the same combination of systems.
Car Dealership Guy has covered the cost of fragmented AI stacks: customer context gets lost, reporting separates, and employees fill the gaps. A product that creates another isolated queue may improve one task while making the operating system worse.
Security, privacy, and governance: 20 points
- What dealership and customer data is collected, and for what purpose?
- Is dealership data used to train shared models or improve products for other customers?
- Which subprocessors receive data, and where is it stored?
- What access controls, encryption, logs, retention, deletion, and incident-response commitments exist?
- Will the vendor sign the security and service-provider terms your program requires?
The FTC’s dealer Safeguards Rule guidance emphasizes selecting capable service providers, requiring appropriate safeguards by contract, and periodically assessing them when covered customer information is involved. Route the answers through qualified security and legal review.
Customer experience and human control: 15 points
- Can the dealership define prohibited claims, approval rules, tone, hours, and escalation triggers?
- How does a customer request a person, correct bad information, or opt out?
- Can a manager pause the workflow immediately?
- Does the product clearly identify the dealership and preserve conversation history for the employee taking over?
Ask to review raw conversations, including failures. A dashboard average cannot show whether one incorrect price or repeated message damaged trust.
Measurement and reporting: 15 points
- Can reporting separate eligible volume, AI-handled work, human-handled work, appointments, shows, sales or repair orders, errors, and opt-outs?
- Can the store export event-level data and reconcile it with the CRM or DMS?
Do not accept vendor-defined “engagement” as the primary KPI. Agree on the outcome and attribution method before launch. Record the baseline yourself.
Commercial terms and exit: 10 points
- What are the complete costs, term, renewal, usage limits, implementation fees, integration fees, support levels, data-export rights, and termination obligations?
Clarify ownership of prompts, workflow logic, transcripts, configured knowledge, reporting history, and derivative material. Require a practical offboarding process: export, revocation of access, deletion timeline, and continuity plan.
Three red flags that should stop the meeting
- The vendor will not describe data use, subprocessors, or deletion.
- The system cannot show source data or conversation history behind important actions.
- The sales team promises business outcomes but will not define eligible volume, attribution, or exclusions.
Another warning is an implementation plan built entirely around the vendor’s team. The dealership needs an internal owner and an operating procedure it can understand.
Run a paid, bounded pilot
A free trial often produces weak accountability on both sides. A small paid pilot with a written scope is cleaner. Define one rooftop, one workflow, one manager, one baseline, one scorecard, and one decision date. Agree in advance on scale, repair, and stop criteria.