Data science portfolio tutorial: when the baseline wins

Practise forecasting, evaluation and an honest project handover with a small runnable example. This tutorial supports the Australian data science career guide.

The records are synthetic and the result is a teaching example. The original run is dated 10 September 2026; this standalone tutorial was assembled on 25 October 2026. Keep those dates with the evidence when you adapt the exercise.

1. Define the portfolio project

Here is an illustrative brief you can adapt: forecast the next fortnight's daily workload for a fictional service team, using a permitted public dataset or explicitly synthetic records. A synthetic exercise demonstrates engineering and reasoning, not real customer demand or savings. The runnable example below uses only invented records.

1. Decision: what action would a forecast inform, and who would check it? Keep automatic staffing or customer actions outside the demonstration.
2. Data: document origin, licence, collection period, units, missing values and permitted redistribution. Publicly accessible does not mean unrestricted use.
3. Baseline: compare with a simple previous-period or seasonal estimate. Explain why that baseline matches the decision.
4. Evaluation: reserve later observations for a time-dependent forecast, and explain whether repeated customers or other groups can leak across the split.
5. Result: report the metric, units, evaluation period and failures. If the model loses to the baseline, report that—it is still a useful result.
6. Handover: include setup commands, pinned dependencies, a small permitted fixture and expected outputs. Do not require private credentials to inspect the core demonstration.

Fit preprocessing only on training data and apply the learned transformations consistently to evaluation data. Do not tune on the final test set. Pipelines help enforce the sequence, but cannot repair a badly chosen split or features unavailable at prediction time. See the scikit-learn guide to leakage and preprocessing.

Run the portfolio example: when the baseline wins

This is a completed local teaching exercise, not a client case or blind benchmark. You need to be comfortable reading JavaScript, using a terminal and checking arithmetic. Node is used to keep this example dependency-free; it is not an employer requirement. There is no provider account, installation of packages, customer data or automated staffing action.

Save forecast.mjs, synthetic-workload.json and forecast.test.mjs in one folder. The README contains the editable checklist and the result file lets you compare every output. Read the files before running them; if your browser adds .txt to a code filename, restore the displayed name.

• forecast.mjs — Required: fitting, forecasting, scoring and command-line runner.
• synthetic-workload.json — Required: all 42 invented daily records.
• forecast.test.mjs — Required for verification: 11 executable tests.
• recorded-result.json — Compare your output with the complete recorded local result.
• README.md — Setup, failure analysis, limitations and editable portfolio checklist.

node --test forecast.test.mjs
node forecast.mjs

Recorded locally on 10 September 2026 using Node 25.2.1 on macOS arm64: 11 tests passed. The script prints its full JSON report; it does not contact a service or save a file. These are automated checks of this version, not independent human review or a guarantee that a different runtime works.

Compare the same future dates, not a flattering screenshot

The 42 synthetic records contain a deliberately changing workload pattern. Fit on 5 January–1 February 2026 (28 days), then forecast 2–15 February (14 days) from the end of 1 February. Both methods keep that same forecast origin; neither updates with observations from the forecast period.

• Baseline: repeat the last training week's count for the matching weekday. The second forecast week still uses training data.
• Candidate: fit a line to four training weekly means, then add each weekday's average training deviation. It estimates a simple trend and weekday pattern, not a language model. Negative estimates would be flagged and clipped to zero; none were clipped here.
• Metric: mean absolute error (MAE)—add the absolute daily errors and divide by the number of evaluated days. It has units of requests/day, not percent accuracy or money saved.

For the method background, see Hyndman and Athanasopoulos on seasonal naive baselines and out-of-sample error and MAE. Those sources explain the methods; they did not test this original fixture or validate its result.

Trace one failure: on 9 February, the invented observed count is 26; the baseline predicts 30 (absolute error 4) and the candidate predicts 35.5 (error 9.5). The training pattern grows, but the constructed evaluation levels decline. A good training fit did not make extrapolation appropriate.

Decision: do not recommend this candidate on this result. Retain the baseline for the exercise and explain the failure. Neither method is approved for a real staffing decision: there are no prediction intervals, representative customer observations, holiday effects, business costs or operational acceptance. Reproducing the files alone is not a job-ready portfolio.

“Held out” here means excluded from fitting, not unknown to the lesson's designer. The full fixture and result are visible. If you change the model after inspecting them, these dates are development evidence—not a fresh test. Plan a training-only validation process and an untouched evaluation period before making a new performance claim.

The supplied tests check a hand-solvable trend, independent error arithmetic, invalid/missing fields, duplicate/gapped dates, forecast overlap, fixed-origin predictions and missing input files. They also prove that altering future counts cannot change fitted values or predictions in this implementation. They do not establish that your replacement dataset or features are free from every kind of leakage.

2. Review your portfolio

Copy the following checks into your repository issue tracker. Record pass/fail, the command or evidence link, and an unresolved limitation for each. This is a practical review aid, not an employer certification.

Ask another person to rerun the documented setup and identify one confusing decision. Record feedback only if the review actually occurs; do not label a self-check an independent review. Remove secrets and confidential records before sharing a repository or sending material to an AI service.

The downloadable portfolio checklist and handover notes include fields for your role evidence, data permissions, source version, results, AI assistance and actual reviewer feedback. For a separate software-delivery task after this evaluation exercise, use the bounded AI-assisted prototype guide. Do not describe either learning lab as paid client experience.

3. Explain your use of AI coding tools

Use assistance for a bounded change: drafting a parser, proposing edge cases or explaining an error. Review the diff before execution, verify dependencies and run tests you understand. Keep a short note of the tool/version, task, accepted changes, rejected suggestions and your checks. Do not publish confidential prompts or raw client inputs.

For an interview or take-home, ask what assistance is permitted before using it. Be ready to explain a failure, alter a test and defend the metric without treating generated text as authority. A polished explanation unsupported by the repository is not delivery evidence.

This reference lab's provenance: an AI assistant drafted its code, synthetic fixture and tests; those tests and the command-line report were run locally. No hiring manager, client or independent delivery reviewer has approved it. When adapting it, identify your own changes, rejected suggestions and checks rather than presenting the unchanged starter as entirely your work.