Analytics & Theme Clustering

Use AI-powered analytics to identify recurring themes, track trends over time, and discover patterns in your lessons learned database.

Overview

Individual lessons capture specific insights, but the real power emerges when you analyze lessons collectively. Cothon's analytics dashboard uses machine learning to identify themes, trends, and patterns that would be invisible when viewing lessons one at a time.

Accessing Analytics

Navigate to Lessons Learned → Analytics tab

The analytics dashboard consists of four main sections:

1. Overview Metrics - High-level statistics
2. Theme Clustering - AI-identified recurring topics
3. Trend Analysis - Changes over time
4. Impact Analysis - Correlation between lessons and outcomes

Overview Metrics

Summary Statistics

Total Lessons:

• Overall count
• Breakdown by category (Win, Loss, Process, Technical)
• Growth trend (lessons per month)

Lesson Distribution:

Visual breakdown showing:

• Category distribution (pie chart)
• Department distribution (bar chart)
• Project type distribution (bar chart)

Engagement Metrics:

• Total views across all lessons
• Average endorsements per lesson
• Average comments per lesson
• Most endorsed lessons (top 10)
• Most viewed lessons (top 10)

Action Item Statistics:

• Total action items created from lessons
• Action item completion rate (%)
• Average time to completion (days)
• Overdue action items count

Contributor Insights:

• Total contributors
• Most active contributors (by lesson count)
• Average lessons per contributor
• New contributors this month

Filters for Metrics

Adjust the time window and scope:

Date Range:

• Last 30 days
• Last 3 months
• Last 6 months
• Last year
• All time
• Custom range

Department:

• All departments
• Specific department(s)

Category:

• All categories
• Wins only
• Losses only
• Process only
• Technical only

Apply filters: Metrics recalculate based on your filter selection, allowing focused analysis.

Theme Clustering

What Is Theme Clustering?

Theme clustering uses unsupervised machine learning (K-means and hierarchical clustering) to group lessons by semantic similarity. It identifies recurring topics without requiring manual tagging.

How it works:

Viewing Themes

The Theme Clusters section displays discovered themes as cards:

Each theme card shows:

• Theme Name - AI-generated label (e.g., "Timeline Estimation Issues")
• Lesson Count - How many lessons belong to this theme
• Prevalence - Percentage of total lessons
• Trend - Increasing, stable, or decreasing over time
• Key Terms - Most common words/phrases in this theme
• Sample Lessons - 2-3 example lesson titles from the theme

Example Theme Card:

┌─────────────────────────────────────────────┐
│ Timeline Estimation Issues                   │
│                                              │
│ 23 lessons (18% of total)                   │
│ Trend: ↑ Increasing                         │
│                                              │
│ Key Terms:                                   │
│ • underestimated (18 mentions)              │
│ • schedule delay (15 mentions)              │
│ • resource constraints (12 mentions)        │
│ • unrealistic timeline (11 mentions)        │
│                                              │
│ Sample Lessons:                              │
│ • Loss: Underestimated Integration...       │
│ • Process: Build 20% Buffer into...         │
│ • Loss: Resource Conflicts Delayed...       │
│                                              │
│ [View All Lessons in Theme] [Export]        │
└─────────────────────────────────────────────┘

Interacting with Themes

Drill down: Click View All Lessons in Theme to see every lesson in that cluster.

Export: Download theme analysis as PDF or CSV for reporting.

Adjust clustering: Change the number of themes (5, 10, 15, 20) to get more granular or broader groupings.

Time-based themes: Toggle "Show temporal themes" to see how themes evolve across time periods.

Common Themes Identified

Based on typical procurement organizations, common themes include:

Theme Evolution Over Time

The Temporal Theme Analysis view shows how themes change:

Timeline view: X-axis: Time (months or quarters) Y-axis: Lesson count per theme

Insights:

Growing themes (upward trend)

• Issues becoming more common
• May indicate systemic problems worsening
• Requires urgent attention

Declining themes (downward trend)

• Issues being successfully addressed
• Action items are working
• Validate and celebrate the improvement

Stable themes (flat trend)

• Persistent recurring issues
• May need different approach
• Consider root cause analysis

Emerging themes (sudden appearance)

• New types of issues
• Environmental changes (new regulations, market shifts)
• May require new processes or training

Example temporal analysis:

Timeline Estimation Issues
  Q1 2025: 3 lessons
  Q2 2025: 5 lessons ↑
  Q3 2025: 7 lessons ↑
  Q4 2025: 8 lessons ↑
  Q1 2026: 4 lessons ↓

Analysis: Timeline issues grew through 2025 but
declined in Q1 2026 after implementing buffer-based
estimating process. Action items appear to be working.

Trend Analysis

Win/Loss Rate Over Time

Track your bidding success:

Chart:

• X-axis: Time period (monthly or quarterly)
• Y-axis: Win rate percentage
• Lines: Overall win rate, win rate by department, win rate by project type

Overlay with lessons: Show lesson creation dates to see if capturing lessons correlates with improved win rates.

Example insight: "Win rate increased from 35% to 48% in the 6 months following implementation of daily standups (captured in Lesson #42)."

Lesson Creation Rate

Track knowledge capture momentum:

Chart:

• X-axis: Time (months)
• Y-axis: Lessons created
• Segmented by: Category, department, or author

Healthy patterns:

• Steady or increasing creation rate
• Lessons created across categories (not just losses)
• Distributed authorship (not just one person)

Concerning patterns:

• Declining creation rate → team losing discipline
• Only loss lessons → not capturing successes
• Single author → knowledge siloed

Action Item Completion Trends

Measure follow-through:

Chart:

• X-axis: Time (months)
• Y-axis: Action item completion rate (%)
• Lines: Overall rate, by priority, by department

Healthy patterns:

• 60%+ completion rate
• High-priority items completed faster
• Improving completion rate over time

Concerning patterns:

• <40% completion rate → no follow-through
• Declining completion rate → team overwhelmed
• High-priority items not completed → prioritization failure

Most Common Tags

Tag frequency over time:

Tag cloud: Size indicates frequency of use

Trend tracking: Watch tags appear, grow, or decline:

• Emerging tags → new focus areas
• Growing tags → increasing issues
• Declining tags → problems being solved

Example: Tag "pricing-strategy" appeared in 2 lessons in Q1, 8 lessons in Q2, 3 lessons in Q3 → suggests pricing was a major issue in Q2 that has since been addressed.

Pattern Identification

Recurring Issues

The Recurring Issues section identifies lessons about the same problem:

Detection method:

• Semantic similarity analysis
• Shared tags or keywords
• Linked to the same root causes

Example:

┌─────────────────────────────────────────────┐
│ Recurring Issue: SME Availability           │
│                                              │
│ 12 lessons mention this issue                │
│                                              │
│ • Loss: SME unavailable for site visit       │
│ • Loss: Technical expert couldn't review...  │
│ • Process: Book SMEs at bid kickoff...       │
│ • Win: Dedicated SME assignment worked...    │
│ • Loss: SME conflicts delayed proposal...    │
│ • Process: Create SME availability calendar  │
│ • [6 more...]                                │
│                                              │
│ Pattern: SME resource conflicts appear in    │
│ 12 lessons across 8 bids over 18 months.    │
│                                              │
│ Action Items Created: 5                      │
│ Action Items Completed: 2 (40%)              │
│                                              │
│ Recommendation: This recurring issue needs   │
│ systematic solution. Consider: (1) dedicated │
│ proposal SME allocation, (2) earlier SME     │
│ booking, (3) SME backup pool.                │
│                                              │
│ [Create New Action Item] [View All Lessons] │
└─────────────────────────────────────────────┘

Benefits:

• Highlights systemic issues that simple action items won't fix
• Shows when a problem keeps appearing despite attempted solutions
• Helps prioritize organizational changes vs. one-off fixes

Success Patterns

The Success Patterns section identifies what consistently works:

Detection method:

• Win lessons with shared themes
• High endorsement counts
• Replicated strategies across multiple wins

Example:

┌─────────────────────────────────────────────┐
│ Success Pattern: Phased Delivery Approach    │
│                                              │
│ Appeared in 8 winning bids                   │
│ Average technical score: 46/50 vs 38/50     │
│                                              │
│ Wins Using This Approach:                    │
│ • ISED Cloud Migration ($2.4M)              │
│ • PSPC System Integration ($1.8M)           │
│ • Health Canada Data Migration ($3.1M)      │
│ • [5 more...]                                │
│                                              │
│ Common Elements:                             │
│ • 3-phase delivery model                     │
│ • Go/no-go gates between phases             │
│ • Rollback procedures                        │
│ • Risk quantification                        │
│                                              │
│ Client Types Where This Works:              │
│ • Departments with past failed projects      │
│ • Risk-averse agencies                       │
│ • Complex integrations (>30 applications)   │
│                                              │
│ Recommendation: Standardize this approach    │
│ in proposal templates for relevant RFPs.     │
│                                              │
│ [Update Templates] [Create Playbook]        │
└─────────────────────────────────────────────┘

Benefits:

• Codifies winning strategies for replication
• Builds confidence in proven approaches
• Enables knowledge transfer to new team members

Failure Modes

The Failure Modes section identifies common reasons for losses:

Detection method:

• Loss lessons with shared themes
• Low technical or pricing scores
• Recurring competitor strengths

Example:

┌─────────────────────────────────────────────┐
│ Failure Mode: Underestimated Complexity      │
│                                              │
│ Appeared in 7 losing bids                    │
│ Average score gap: -12 points vs winner     │
│                                              │
│ Losses With This Pattern:                    │
│ • Healthcare Integration RFP (2024-08)      │
│ • Provincial ERP Migration (2024-11)        │
│ • Federal Mainframe Modernization (2025-01) │
│ • [4 more...]                                │
│                                              │
│ Common Characteristics:                      │
│ • Legacy systems (>15 years old)            │
│ • "Integration" in project scope            │
│ • Aggressive timeline proposed               │
│ • Insufficient discovery phase              │
│                                              │
│ Root Causes:                                 │
│ • Estimating multipliers too low (1.5x vs   │
│   required 3x for mainframes)                │
│ • No discovery phase in scope               │
│ • Pressure to be cost-competitive           │
│                                              │
│ Recommendation: For legacy integration bids, │
│ mandate 3x complexity multiplier and         │
│ discovery phase. Price realistically even if │
│ less competitive—credibility matters more.   │
│                                              │
│ [Update Estimating Guidelines] [Create Alert]│
└─────────────────────────────────────────────┘

Benefits:

• Prevents repeating the same mistakes
• Identifies high-risk bid characteristics
• Informs go/no-bid decisions

Department-Specific Patterns

Analyze lessons by government department:

View: Select a department (e.g., ISED, DND, Health Canada) to see:

• Win rate with that department
• Common winning strategies
• Common failure modes
• Typical scoring patterns
• Evaluator preferences (inferred from lessons)

Example:

Department: Innovation, Science and Economic Development (ISED)

Win Rate: 42% (8 wins, 11 losses)

Winning Patterns:
• Innovation emphasis (7/8 wins mentioned innovative approach)
• Phased delivery (6/8 wins)
• Past performance with ISED (5/8 wins)

Common Loss Reasons:
• Pricing too high (6/11 losses)
• Weak innovation story (5/11 losses)
• Competitor with ISED past performance (4/11 losses)

Evaluator Preferences (inferred):
• Technical score weights heavily (avg 60% of total)
• Innovation and risk mitigation highly valued
• Past performance with ISED is a major differentiator

Recommendations:
• Lead with innovation in executive summary
• Reference past ISED projects prominently
• Price competitively—ISED is price-sensitive
• Emphasize risk mitigation for complex projects

Use case: Before bidding on a new ISED opportunity, review department-specific patterns to tailor your approach.

Impact Analysis

Correlation Analysis

Identify which factors correlate with success:

Win Rate by Characteristics:

Insights:

• Phased delivery approach correlates with 73% win rate vs 38% baseline
• SME past performance is a major differentiator
• Win rate declines with contract value (likely due to increased competition)
• Shorter timelines have higher win rates (possibly indicates better project fit)

Action Item Impact

Measure whether completed action items improve outcomes:

Analysis: Compare win rates before and after action items were implemented.

Example:

Action Item: Implement daily standups for proposals >50 pages
Created: 2025-01-15
Completed: 2025-02-01

Bids Before Implementation (>50 pages):
• Count: 12 bids
• Win Rate: 33% (4 wins, 8 losses)
• Avg proposal revisions: 5.2

Bids After Implementation (>50 pages):
• Count: 8 bids
• Win Rate: 50% (4 wins, 4 losses)
• Avg proposal revisions: 3.1

Impact:
• Win rate improved by +17 percentage points
• Revisions reduced by 40% (5.2 → 3.1)

Conclusion: Action item appears to have positive impact.
Continue practice and monitor over larger sample size.

Caveats:

• Small sample sizes (< 20 bids) may show random variation, not real impact
• Other factors may have changed simultaneously
• Use as directional guidance, not proof

Lessons with Highest Impact

Identify which lessons drove the most change:

Ranking criteria:

• Endorsement count (team validation)
• Number of action items created
• Action item completion rate
• Measurable impact (if available)
• Citations in subsequent bids

Example Top 5:

Use case:

• Onboard new team members with the top-impact lessons
• Recognize authors of high-impact lessons
• Prioritize similar lessons for action item creation

Exporting Analytics

PDF Reports

Generate formatted reports for stakeholders:

Quarterly Lessons Learned Report

• Executive summary of key metrics
• Theme clustering visualization
• Top lessons by impact
• Action item progress
• Recommendations for next quarter

Department-Specific Report

• Lessons from specific department
• Win/loss patterns
• Success and failure modes
• Tailored recommendations

Custom Reports

• Choose date range, filters, sections to include
• Add custom commentary
• Annotate themes and patterns

Access: Click Export → Generate PDF Report → Select report type → Download

CSV Data Exports

Export raw data for advanced analysis:

Available exports:

• All lessons (full text and metadata)
• Theme cluster assignments
• Action items with status
• Win/loss data with characteristics
• Tag frequency and co-occurrence

Use cases:

• Import into Excel or Tableau for custom visualization
• Perform statistical analysis in R or Python
• Integrate with other business intelligence systems

Access: Click Export → Export to CSV → Select data type → Download

Dashboard Sharing

Share live dashboards with stakeholders:

Create shared dashboard:

1. Configure filters and views
2. Click Share Dashboard
3. Set permissions (view-only or interactive)
4. Generate shareable link or embed code

Use cases:

• Executive dashboard for monthly leadership meetings
• Department dashboards for team leads
• Public dashboard for organizational transparency

Advanced Features

Custom Clustering

Override AI-detected themes with manual clustering:

When to use:

• AI themes don't align with your mental model
• You want themes organized by a specific taxonomy
• Combining or splitting themes for clarity

How:

1. Enable Advanced Mode in theme clustering
2. Drag lessons between theme clusters
3. Rename themes as needed
4. Lock themes to prevent AI from reclustering

Predictive Analytics (Beta)

Use historical lessons to predict future outcomes:

Prediction models:

• Win probability - Given bid characteristics, predict likelihood of win
• Score prediction - Estimate likely technical and pricing scores
• Risk factors - Identify high-risk aspects of a new opportunity

How it works: Train models on your historical lessons + bid outcomes, then apply to new opportunities.

Learn more: Risk Predictions

Anomaly Detection

Identify unusual lessons that don't fit patterns:

Anomalies:

• Unexpected win despite unfavorable factors
• Surprising loss on seemingly good bid
• Outlier lessons that don't cluster with others

Why it matters: Anomalies often contain the most valuable insights—unique situations that require special attention.

Example: "Win on $5M contract despite pricing 15% higher than usual—evaluators prioritized risk mitigation over cost. Suggests new strategy for high-stakes bids."

Natural Language Queries (Experimental)

Ask questions in plain language:

Examples:

• "What are our biggest weaknesses on ISED bids?" → AI analyzes ISED losses and summarizes common failure modes
• "Which team member has the most expertise on cloud migration?" → AI identifies authors of cloud migration lessons and their contributions
• "How has our win rate changed since implementing daily standups?" → AI compares win rates before and after action item completion date

Access: Click Ask a Question in the analytics dashboard, type your query, get AI-generated insights.

Best Practices

Review Analytics Monthly

Make it routine: Schedule a monthly analytics review session with your team.

Agenda:

• Review new themes that emerged
• Track action item completion rates
• Celebrate lessons with high impact
• Identify recurring issues needing systematic solutions

Share Insights with Leadership

Quarterly executive briefings: Generate PDF reports showing:

• Win rate trends
• ROI from lessons learned program (wins attributed to lesson-driven improvements)
• Key themes and recommended organizational changes

Use case: Justify continued investment in knowledge management by demonstrating measurable impact.

Use Themes for Strategic Planning

Annual planning: Review theme clusters to identify strategic priorities:

• Which themes appear most frequently in losses? → Areas needing investment
• Which success patterns are we underutilizing? → Quick wins to replicate
• Which departments show highest win rates? → Model for others

Track Theme Evolution

Quarterly trend check: Review temporal theme analysis to see what's improving and what's worsening:

• Declining negative themes → Validate that action items are working
• Growing negative themes → Escalate for urgent attention
• New themes → Investigate root causes

Benchmark Against Goals

Set targets:

• Lesson creation: 2-3 per completed bid
• Action item completion rate: >60%
• Endorsement rate: >40% of lessons endorsed
• Win rate improvement: Track annually

Review progress: Use analytics dashboard to measure actual performance vs. targets.

Frequently Asked Questions

Next Steps

Now that you understand analytics:

• Explore Risk Predictions - Apply lessons to predict future risks
• Configure Review Gates - Set up quality control workflows
• Advanced Analytics - Cross-module analytics and reporting