Starting an AI startup: validate the problem and test your first delivery

If you use AI coding tools and are considering an AI startup, separate two questions: does someone need the product, and can you deliver the promised workflow? A polished demo answers neither on its own. This guide helps an early-career builder investigate the first question and reproduce a small delivery review for the second.

You do not need to found a company to learn delivery skills or apply for scoped project work. Start with the uncertainty that actually blocks you. The example below is an invented event-reply prototype, not an MLAI client project or event service.

The recorded exercise was run on 10 September 2026 using Node 25.2.1 on macOS arm64. It retains the failed requirement and reproducible outputs. It has not received independent recipient acceptance.

1. Choose a problem you can investigate

Write the user, current task and costly interruption before selecting a model. For example: “A club organiser finds an event notice, checks its timezone, then drafts a reply.” That is a research hypothesis—not proof that the organiser needs a chatbot.

Ask a consenting potential user about the last time they did the task: what started it, which information they used, what failed and how they recovered. Ask to inspect a permitted, anonymised example if appropriate. Compare the proposed workflow with their current search, saved reply or manual process. Record what you observed separately from your interpretation.

A useful contradictory finding might be that an existing template already solves the problem. In that case, stop expanding the AI prototype and investigate the remaining difficulty. Do not count compliments about a demo as payment, adoption or validated demand.

2. Keep demand and delivery evidence separate

On a small screen, scroll the table sideways to compare the evidence paths.

For a broader customer-research exercise, use the founder validation guide. For the model's mechanics and evidence boundaries, use the prototype implementation guide. Here, the distinct task is deciding whether the delivered behaviour meets a brief.

3. Define the first delivery before coding

Fictional brief: given one event ID, one question and a notice version, draft the time or location from consistent, permitted notices for an organiser to review. If the question is unsupported, abstain. Refuse personal-data requests. Preserve the stated timezone and supporting notice IDs.

• Inputs: three invented notices, 12 training questions and eight selected development cases. No real attendee records or customer conversations.
• Output: a fixed-template draft copied from structured notice fields, with citations and a human-review flag; otherwise no draft.
• Excluded: inbox integration, sending, booking, payment, attendee lookup and ongoing operation. No external API, LLM subscription or API key is involved.
• Failure handling: conflicting/missing notices, a stale snapshot or classifier failure must not produce an invented answer. The original lab tests exercise these boundaries.

The model is a small learned word-count classifier compared with separate keyword rules. This is an AI-assisted coding exercise, not a large-language-model benchmark. A form with explicit time/location choices might avoid the classifier altogether; test that simpler alternative against the user's task.

4. Review the actual result—not just the test badge

The downloaded kit has 22 passing code tests: 14 for the prototype and eight for the delivery review. Some tests deliberately verify known failures. Passing them means the program behaved as recorded, not that it satisfied the brief.

Against these eight selected application requirements, the rules matched 5 / 8 and the classifier matched 7 / 8. The delivery decision is HOLD. These invented development cases are visible to the author; the counts are not representative accuracy or independent evaluation.

Scroll the table sideways on a small screen to compare both implementations.

Why the email and parking cases matter

For q6, both classifiers incorrectly choose location for an attendee-email question. The earlier application boundary refuses the request, so the application requirement matches while the raw classification remains wrong. Raw intent routing is still only 4/8 for rules and 6/8 for the learned classifier.

For q8, both produce “Start: 18 February 2026, 18:00 Australia/Melbourne.” The citations genuinely support that time—but the question asked about parking. A valid citation and “needs human review” label do not make an irrelevant draft useful. Both implementations fail this requirement.

The rules also abstain on q3 and q4, which the brief expects them to answer. Those are missed supported tasks, not data leaks. Distinguish non-completion, wrong answers and unsafe actions rather than hiding them inside one success rate.

Next decision: do not accept this delivery for the stated brief. Investigate the relevance failure, compare a narrower interface and add separately reviewed paraphrases and negations before changing the code. Preserve this v1 result. Matching the eight known questions later would still require review, not automatic release.

5. Download and reproduce the review

Download the complete first-delivery kit (ZIP). Extract its ten files into one empty folder. Inspect the code before running it. No package installation or network access is needed by the programs.

node --test lab.test.mjs delivery-check.test.mjs
node lab.mjs
node delivery-check.mjs

The first command should report 22 passing tests; the second should reproduce recorded-result.json. The last prints delivery-record.json and deliberately exits with code 2 because its decision is HOLD. Exit 1 indicates an error. Do not suppress the refusal to accept the delivery.

This reuses the prototype guide's original lab, not a second independent experiment. Its fixed February 2026 clock and notices are historical teaching inputs, not current event information. The separate delivery review checks exact drafts/citations against a fixed brief and rejects missing, altered or duplicated cases and inconsistent counts. It is not a general-purpose semantic evaluator.

6. Hand over the decision and limitations

A recipient needs enough information to reproduce the result and decide what can happen next. The kit's DELIVERY.md contains a completed record, not just blank headings. Its main decisions are:

Accepted scope

Teaching demonstration only; this is not acceptance of the fictional client brief.

Evidence delivered

Source, invented fixture, 22 code tests, complete experiment output and the HOLD acceptance review.

Unresolved defect

The parking request gets a time draft. A delivery owner and independent reviewer still need to assess any repair and new cases.

Disable and fallback

Do not connect it to real communication channels. Stop running it and consult the original notices manually. Switching to the rules baseline does not repair q8.

Commercial status

No client, quote, payment, support commitment or approved operational deployment. Permission, ownership and support terms remain unresolved.

Illustrative capacity: 6 hours building + 4 testing + 2 documenting + 2 reviewing changes is 14 hours total. With 8 hours available, the gap is 6 hours. Narrow the scope or change the schedule; do not quietly remove testing or handover. These hours are invented, not actual time logged or a price estimate.

If you already run a startup and are weighing client work alongside it, use the founder capacity and pilot guide to account for weekly commitments and later support separately.

Keep a separate customer-investigation record

Download the editable problem-validation worksheet. It separates observation, interpretation, contradictory evidence, the smallest test and the next commitment. Its fictional customer scenario remains UNRUN: executing this technical kit does not establish an interview, market demand or willingness to pay.

For your own project, retain consent and data permissions, the actual observed task, the alternative tested, dated results and a reason to continue, change or stop. Record unknowns explicitly. Keep personal details and private business notes out of public portfolio copies.

Check operating obligations alongside validation

For activity in Australia, use the Australian Government starting guide and registration guidance to identify requirements for your circumstances. Customer validation is not a universal registration trigger. An experimental label does not establish an exemption.

Check applicable registrations, licences, tax, data and contractual obligations before operating or making commitments; seek qualified advice where needed. This guide does not determine your obligations or cover New Zealand law.

Bring your own delivery evidence to the next step

Keep evidence honest: a generated test is not a passing test, a mock is not a live integration, and a synthetic project is not client experience. Explain your own changes, AI coding tools used, what you manually reviewed, actual test results and unresolved limitations. Do not present this supplied starter as your original client work.

If you are still learning, reproduce the exercise and bring a question to a relevant MLAI event; check its topic, prerequisites and online or local format. If you can demonstrate broader delivery ability and want scoped paid work, the Studio application below is the appropriate next step. It does not guarantee a project or job.

Share only permitted repository/demo material and realistic availability. The public intake is Australia-focused; New Zealand contractor eligibility remains unverified. Client permission is needed before assuming their work can enter a public portfolio.