Day 2

Context

Tasks (4)

1. Tokenize your content using the real API (25 min)
   Use the Anthropic token counting endpoint (POST /v1/messages/count_tokens) to count exact tokens for: your resume, a product spec, and a customer support email. Compare the actual count to the rough estimate (words ÷ 0.75). How far off is the estimate? How would this affect a cost model at 10,000 requests/day? Save as /day-02/token_analysis.md.
2. Run temperature experiments (25 min)
   Using the Claude API (model: claude-sonnet-4-6), send the same prompt ("Write 3 product names for an AI coding assistant") at temperature 0, 0.5, and 1.0. Document the differences. When would you use each setting in a real product? What does Anthropic recommend about top_p and top_k? Save as /day-02/temperature_experiments.md.
3. Design a context window budget (25 min)
   You are building a document Q&A product. Users upload up to 3 documents (~50K tokens each). Design the context budget: system prompt (how many tokens?), documents (which ones fit?), conversation history (how many turns?), tool results (how much space?). What is cut first when you hit 200K? When does the product need RAG vs full-context? Consider: Gemini 2.5 Pro at 1M+ removes the chunking need — but at what cost? Save as /day-02/token_budget_design.md.
4. Explain sampling to a non-technical stakeholder (25 min)
   Write a 150-word explanation of why your AI product sometimes gives different answers to the same question. This is a real customer objection you will face. Make it accurate, simple, and reassuring. Save as /day-02/customer_explanation.md.

Token counting and cost framework — JavaScript

// Day 02 — Token Counting and Context Budget Framework
// IMPORTANT: Use Anthropic's token counting API in production, not estimates.
// API endpoint: POST https://api.anthropic.com/v1/messages/count_tokens
// Docs: https://docs.anthropic.com/en/api/messages-count-tokens

// For rough planning: approximate estimator (NOT for production use)
function roughEstimateTokens(text) {
  // Word-based: within 10-25% of actual BPE count
  // WARNING: Inaccurate for code, non-English, structured data
  const words = text.trim().split(/\s+/).length;
  return Math.ceil(words / 0.75);
}

// Context window budgets by model (verify at docs.anthropic.com/en/docs/about-claude/models)
const CONTEXT_WINDOWS = {
  'claude-sonnet-4-6':       200000,  // 200K combined input+output
  'claude-opus-4-6':         200000,
  'claude-haiku-4-5-20251001': 200000,
  'gpt-4o':                  128000,
  'gemini-2.5-pro':         1000000,  // 1M+ tokens
};

// Context budget designer
function designContextBudget(model, systemPromptTokens, maxDocTokens, maxHistoryTurns, avgTurnTokens, reservedOutput) {
  const totalWindow = CONTEXT_WINDOWS[model];
  if (!totalWindow) return { error: 'Unknown model — check docs.anthropic.com' };

  const historyTokens = maxHistoryTurns * avgTurnTokens * 2; // user + assistant per turn
  const totalNeeded = systemPromptTokens + maxDocTokens + historyTokens + reservedOutput;
  const headroom = totalWindow - totalNeeded;

  return {
    model,
    totalWindow,
    budget: {
      systemPrompt: systemPromptTokens,
      documents: maxDocTokens,
      conversationHistory: historyTokens,
      reservedForOutput: reservedOutput,
      total: totalNeeded,
      headroom: headroom
    },
    fits: headroom > 0,
    recommendation: headroom > 0
      ? 'Full-context approach viable — no RAG needed'
      : 'Over budget by ' + Math.abs(headroom) + ' tokens — need RAG or document selection'
  };
}

// Scenario: Document Q&A product
console.log('CONTEXT WINDOW BUDGET — Document Q&A Product');
console.log('='.repeat(60));

const budget = designContextBudget(
  'claude-sonnet-4-6',
  2000,    // system prompt
  120000,  // 3 docs × 40K avg
  5,       // 5 turns of history
  200,     // ~200 tokens per message
  4000     // reserve 4K for output
);

console.log('Model:             ' + budget.model);
console.log('Total window:      ' + budget.totalWindow.toLocaleString() + ' tokens');
console.log('System prompt:     ' + budget.budget.systemPrompt.toLocaleString());
console.log('Documents:         ' + budget.budget.documents.toLocaleString());
console.log('History (5 turns): ' + budget.budget.conversationHistory.toLocaleString());
console.log('Reserved output:   ' + budget.budget.reservedForOutput.toLocaleString());
console.log('Total needed:      ' + budget.budget.total.toLocaleString());
console.log('Headroom:          ' + budget.budget.headroom.toLocaleString());
console.log('Fits?              ' + budget.fits);
console.log('Recommendation:    ' + budget.recommendation);

console.log('\n' + '='.repeat(60));
console.log('SAMPLING PARAMETERS — Anthropic Guidance');
console.log('Temperature: the ONLY parameter you should usually change');
console.log('  0.0 → deterministic (extraction, classification, code)');
console.log('  0.3-0.7 → balanced (conversation, creative tasks)');
console.log('  1.0 → maximum variety (brainstorming)');
console.log('top_p & top_k: Anthropic recommends leaving at defaults');
console.log('\nProduction tip: always use the token counting API, not estimates');
console.log('POST /v1/messages/count_tokens → exact count before making the call');

Interview question

What is a context window and why does it matter for product decisions?

PM angle

Resources

• DOCS Anthropic Token Counting API — Count exact tokens before making API calls. Essential for cost management.
• DOCS Claude Context Windows — Anthropic’s guidance on managing large contexts effectively.
• DOCS Anthropic Prompt Caching — Cache system prompts for 90% cost reduction. Introduced here, deep-dived on Days 9 and 17.
• TOOL OpenAI Tokenizer — Visual BPE tokenizer. Useful for learning — but GPT and Claude tokenizers differ.
• PAPER BPE Tokenization Explainer — How byte-pair encoding works — go as deep as you want.
• DOCS Claude Models Overview — Current context limits, model strings, and capabilities.