AI Bits #4: Do AI coding tools save delivery time?

This issue is for early-career builders using AI coding tools who want to demonstrate dependable project delivery. It revisits two earlier studies, then gives you a filled teaching log, editable worksheet and a runnable exercise for recording your own evidence. It is not a current model ranking or a claim that one workflow improves every developer’s productivity.

Editorial revision, 9 September 2026: the original February issue date is retained. Broad explanations of when AI helps have been replaced with study-specific findings and an explicit measurement plan. Unverified tool endorsements have been removed.

10 September update: added downloadable records, explicit missing-time handling and a checked worksheet; clarified the studies' time and activity measures. The filled CSV example is fictional; it is not a new experiment or client case.

Two findings, two different measurements

In Becker and colleagues’ 25 July 2025 version, 16 experienced open-source developers completed 246 tasks in familiar mature projects. Tasks were randomly assigned to allow or disallow February–June 2025 AI tools, primarily Cursor with Claude 3.5/3.7 Sonnet. The authors report 19% longer completion time when AI was allowed. Section 2.3 measures developers' self-reported implementation work before and after PR review, with some missing post-review times imputed—not calendar duration or every reviewer's labour. The estimate adjusts for forecast task difficulty; Appendix D.2 explains its 95% confidence intervals and sensitivity to the uncertainty method. It is not a fixed slowdown for every developer or today's models.

Xu and colleagues’ 28 January 2026 version analyses open-source activity following Copilot's introduction. Its subgroup results report 19% fewer commits and 6.5% more PR reviews for core contributors, with 95% confidence intervals plotted in Figure 5. These are activity counts—not measured hours, lines of code reviewed or the same completion-time measure as the randomised trial. The methods compare changes across supported and comparison language groups using difference-in-differences; individual Copilot usage is not observed. The result depends on exposure, comparison and contributor-group assumptions, not simply knowing that a person used AI.

These are checks of the abstracts and selected measurement sections, not independent reproduction or a full causal-methods review. Do not average their percentages or call them a combined “AI slowdown rate”. Treat the worksheet below as a way to document your own work, not an attempt to reproduce either study.

Define what “faster” means for your project

Before a small, permitted task begins, specify the behaviour to deliver and the evidence required for acceptance. For example, a fictional CSV import change might need to reject missing required columns, preserve valid rows and explain errors without exposing private data. Merely producing the parser would not complete that task.

On narrow screens, swipe tables sideways; keyboard users can focus a table and use the arrow keys.

A faster draft can still mean more total effort

Fictional teaching example—not a study result or MLAI client measurement: compare two hypothetical delivery records for the same stated acceptance criteria.

Authoring alone fell by 35 minutes, or 43.75%. Total recorded effort increased by five minutes, or 5%. That does not imply a 5% longer calendar duration: people may work concurrently or wait in a queue. Software cost, later defects and ongoing maintenance are outside this example, so it is not a profitability comparison.

This arithmetic does not show that AI caused the difference. Real comparisons need comparable scope, experience and conditions; repeating the same task can introduce learning effects. A single delivery record is useful evidence about work performed, not a controlled experiment or a universal rate.

Use an AI-assisted delivery effort record

The filled fictional delivery record supplies the task, three acceptance criteria, an invented review correction and a HOLD decision because a privacy check remains unrun. Its 105 person-minutes, 240 elapsed minutes and 120 queue minutes are all invented. Queue time is contained in elapsed time; neither is added to active effort.

Download a blank delivery record to use offline, or edit and save the worksheet below. The CSV example supplies no parser or real test results. Use the next section's existing runnable lab if you want to practise with actual commands.

Keep an explicit “not recorded” value when time is missing. Do not replace it with zero or reconstruct precise minutes from memory. Distinguish active attention from tool latency; record how you treated waiting so someone else can interpret the total.

For example, clear the fictional reviewer minutes. The known subtotal becomes 70 minutes, while the complete total becomes unavailable—not 70 or 105. An explicitly recorded zero is different. Even complete fields cannot reveal omitted work or establish that the task was accepted.

Practise on a permitted, reproducible task

Use the environment handover kit to practise recording evidence. Its preserved local run reports 24 passing code tests and a classification-review HOLD. Those are actual local execution results on a synthetic teaching fixture, not evidence of saved author time. No author/reviewer effort measurement or independent recipient approval was recorded for that run.

1. Start a blank log before working. Define one small task and acceptance conditions; record the kit/source version, permitted inputs and chosen tools.
2. Record active work by person and category as it happens. If you revisit the same problem after feedback, put that time in rework once. Keep tool waiting and review queues separate from attention.
3. Follow the kit's actual commands, retain complete outputs and the expected HOLD, and note first failures before changing anything. A code test reproducing a defect is not acceptance of that defect.
4. Ask a reviewer to record their own time and decision. If no reviewer is available, leave their minutes and approval not recorded; do not insert a synthetic reviewer into your real log.
5. Save your source identity, test evidence and handover with the log. One run describes that work; comparing repeated runs on a task you have learned does not isolate the effect of AI.

Turn the record into credible build evidence

1. Choose a permitted example with a bounded task; do not experiment on a client’s production system without authorisation.
2. Use AI coding tools within the agreed data and action boundaries. Inspect proposed changes and test against the task, not just the model’s suggested tests.
3. Provide a short handover: what changed, exact run/check instructions, expected results, remaining failures and the actual review decision. Name missing review instead of inventing acceptance.
4. Make security review proportional to the task: permissions, secrets/data boundaries, input handling and allowed actions. Record who checked them and what is untested; this rubric is not a security certification.
5. Describe your contribution honestly. A worked example can demonstrate your process without claiming an unmeasured speed improvement.

When you have your own permitted build—not just the supplied teaching example—show what you shipped, how you checked it and what another person needs to maintain it. Studio's public positioning is Australian; New Zealand contractor eligibility remains unverified here. Confirm eligibility before assuming coverage. Portfolio use needs permission, and paid-project allocation is not guaranteed.

If you are exploring the topic rather than applying to build, choose a relevant MLAI event. Your worksheet is not automatically attached to either destination.

Sources and limits

The two linked paper abstracts, version dates and selected measurement sections were checked on 10 September 2026. Their exact data and estimates have not been independently reproduced here. The CSV comparison and filled record are fictional editorial teaching material; the linked handover record describes a separate actual local run on synthetic inputs. No new productivity experiment was run. Neither establishes the safest workflow, a hiring outcome or an AI coding tool’s current performance. Independent technical/source review remains outstanding.

Frequently Asked Questions