Lessons Learned: How to Compare Evidence Across Projects

Lessons learned are observations from completed work that may guide a future decision. A useful lesson names the situation, the evidence, what changed, and the conditions under which the next team should try the same move.

Three fictional projects report that a launch checklist helped staff catch missed handoffs. A fourth used it but still missed a handoff. The person in charge was absent.

The phrase “use a checklist” hides that last case. A better lesson asks who must sign off, when, and how the next team will see whether the step helped.

Atlas can compare selected reports and show the cited passage behind each proposed pattern. People still decide which records belong, whether the next setting is similar, and what action to try.

What lessons learned should contain

A lesson is more than a description of what went wrong. It connects experience to a future choice. NASA's lessons learned lifecycle separates collecting, recording, sharing, and applying lessons. A note in a repository has not yet changed a team's practice.

Each entry needs a specific condition, an observed result, a source, and a proposed action. It should also name an exception or boundary. For example, “Add a decision owner to the launch checklist when approval crosses teams” is more useful than “communicate better.” The next team can test the first instruction.

An after action review can yield a first lesson from one event and its team. A cross-project register compares that claim with other projects and notes conflicts. NASA's public lessons system keeps a driving event beside each recommendation. Its reviewed entries still need a fit check before another team uses them.

A lesson can be positive. “Keep the rehearsal call” may be worth preserving if the record shows a smoother handoff and the team can say what the call changed. Use the same evidence standard for a practice to repeat and a practice to alter.

Gather project records with their context

Keep the source trail

Start with a small set of records the team may use. A project review, dated log, plan, and case note each answer a different question.

For each source, keep its project, date, phase, team, and limits. A future team needs that trail to judge fit.

Ask people who did the work to check the written record. A closeout note may omit a workaround or claim that everyone agreed when they did not.

Keep a person's account apart from a dated log. A research source-analysis workflow uses the same habit of tracing each claim to its document.

Capture while work is fresh

Capture a first lesson while details are fresh. NASA's collect phase includes team talks held during the work.

Mark whether the team checked the claim after the result was known. An early note needs that status so readers do not treat it as an approved rule.

A short intake card can hold the project ID, event, result, possible reason, source page, reviewer, access rule, and open question. You need only the files that can test the lesson.

Keep private records under the access rules that govern the work.

PMI's practitioner guide also treats sharing as part of the lesson process. Plan who can read the final entry when you gather it.

Compare claims and counterexamples

Look for a repeated pattern

This evidence check asks what each source shows and where the proposed lesson fails. Start with a question: “When do checklists prevent handoff errors?” Make a row for each project. Note the practice, who used it, the result, and the source. Several reports that copy one summary still count as one account.

PMI's study of project learning calls for teams to gather, sort, and follow up on lessons from many projects. The follow-up shows whether a change helped. A qualitative synthesis workflow can aid the reading step. Keep the source records for the project choice.

Keep exceptions in view

An exception may show that the lesson's condition is too broad. If the same checklist helped three teams and failed in a fourth, ask what differed before discarding either account. Was approval centralized? Was the owner named? Did the schedule change? A plausible rival explanation should stay visible in the record.

GAO's review of a lessons process found that collecting lessons can leave open whether they are sound and where they apply. A tidy list can conceal those questions. Keep a conflicting case in view; it may reveal a limit on the claim.

Separate observation from recommendation

“Two handoffs were missed” is an observation. “No owner was named” may be a documented condition. “Assign one owner before launch” is a recommendation. Link each level to its basis and say where the explanation remains uncertain. If a source is only a participant recollection, identify it as such.

A source table works best when readers can return to the passage. Keep a page, time, or row. PMI's project-knowledge paper warns that stored lessons may never be used. A checked register should help someone act at the next project start.

Read a fictional cross-project register

Compare the cases

This example is fictional. Projects A through D, their source IDs, dates, and results were invented to show how to compare a recurring claim with a counterexample. They do not describe Atlas customers or measured product outcomes.

Four event teams each needed a sponsor's sign-off before posting a public plan. The first three used a launch checklist and reported fewer late requests. The fourth also used a checklist, yet posted the plan before the person in charge replied. The first lesson, “a checklist prevents missed sign-offs,” is too broad.

Table 1: The register keeps the pattern and the failed case in view. It makes no claim that the checklist caused a measured gain.

B has one timing record, C has a person's account, and D shows that a checked box did not get a reply. These sources answer distinct parts of the question.

Turn the pattern into a test

A defensible lesson would read: “For event plans that need a sponsor's sign-off, name the owner and due date on the launch checklist. Keep the reply before posting. Projects A to C suggest the prompt may help. D shows that a checked box did not get a reply. Count missed sign-offs at the next similar event.”

That statement gives the next team an action and a way to learn. A new sponsor process or urgent release may need another rule.

The next team may reject the lesson after checking fit. It should record why.

Write a lesson to test

Keep the entry specific

Use this short form: In [setting], [source-backed finding] suggests [action]. Check [measure] in the next [similar setting]. Add a note about what remains unsure.

This form gives the next team a way to test the lesson and revise it when the next case differs. A practical entry has six fields:

1. Situation: Project phase, people, constraints, and decision at issue.
2. Evidence: Source IDs and the observations they actually contain.
3. Exception: A counterexample, missing record, or setting where the action may fail.
4. Proposed action: The change a team can perform, with an owner who can accept it.
5. Check: What the next project will observe and when.
6. Review date: When to revise, keep, or retire the lesson.

Put a checked step in the work

PMI's paper on project knowledge calls for useful changes in checklists and work plans. NASA's apply phase also calls for changes in practice.

An owner must still weigh cost, risk, and fit before making a rule.

Match the wording to the proof. This fictional register supports “may reduce missed sign-offs in similar events.” It cannot support a claim that the step always works.

If the result calls for a formal study, plan that study. The register does not estimate a causal effect.

Choose a format people will reuse

A closeout report keeps the full story, but may be hard to find when a new team starts. A register helps the team find similar cases.

A kickoff checklist puts one approved step in the work itself. Choose the format that fits the use.

Table 2: NASA's lessons system holds the record. Its lifecycle guide also calls for later use. PMI warns that a store of lessons can go unread.

Put an approved lesson where the next team makes its choice. Keep the source record near enough to check.

A team can use more than one format. The full record may stay in a private project space. A checked summary can go in a register, and an approved step can go in a checklist.

Search ease does not justify copying private detail into a wider space.

Decide which lesson applies

Choose a past lesson when the new task, limits, and roles match the old case. Ask who will do the step and what could make it fail here.

If the work has a legal sign-off rule, use that rule. A team checklist cannot replace it.

Ask an owner to accept or reject the action. Record the choice and the reason. A rejected lesson can show where the old setting differed. GAO's finding makes the fit check part of the work.

Set a check before the project begins. If the aim is fewer missed sign-offs, define a miss and count it. Note any other changes too. For a formal program review, evaluation questions can tie the check to the program's purpose, users, and data that can be gathered.

If the result differs from the hope, revise the lesson. The register should show what the team learned over time.

In an evaluation setting, use the team's formal follow-up process too. The WHO evaluation handbook covers using findings and tracking recommendations after a review.

Check selected reports in Atlas

Atlas can help compare the reports a team has chosen and may use. Add the project reviews, logs, and case notes. Name them in a chat request: “For each claim about sign-off handoffs, show the source text, project setting, result, and any case that differs. Mark what the sources leave open.”

Open each citation and read nearby text. If an answer treats an interview as a log or blends two projects, fix the table. Check whether two reports copied one statement. Save a note only after the project owner has checked the lesson. Public Atlas guides support this source, citation, and note flow; the brief links those guides for source checking.

This first-party screenshot shows an answer beside an opened citation. The visible paper concerns AI science; it is not one of the fictional project records and does not show a generated organizational lesson.

Atlas sees the sources the user selects. The reviewer must choose cases, hear the right people, guard private data, and set the next check. A source comparison guide can help with reading. The people doing the next project own the choice.

Limits of a lessons learned register

Teams tend to save cases they recall. Quiet wins and common failures may be missing. A pattern in the reports may reflect what teams chose to write down.

State how many cases back the claim and what kinds of work they cover.

Records can disagree. One closeout may call a sign-off “late.” Its log may show that the request went out on time but the reply did not. Keep both facts.

Ask the people involved to check the account, above all when a new step may change roles or workload. A generated summary cannot settle staff or policy questions.

Old conditions may no longer hold. A lesson from one steady sponsor may fail when the sponsor role rotates. New rules, staff, scale, or tools may also change the result.

NASA's reviewed lesson entries retain the driving event. The next team still has to judge fit.

The strongest lesson is one a future team can test, revise, and trace back to its evidence. Keep its condition, exception, action, owner, and follow-up together. If the next project learns something different, update the record rather than protecting the first wording.