Claude Code के साथ A/B Testing कैसे Implement करें

A/B Testing Implementation का पूरा Overview

A/B testing data-driven decision making के लिए अनिवार्य है। हालांकि, सही statistical processing, user allocation, और results tracking को खुद implement करना काफी मुश्किल है। Claude Code का उपयोग करके, आप statistically सही A/B testing infrastructure को efficiently बना सकते हैं।

Test Allocation Engine

> Users को consistently A/B groups में allocate करने वाला engine बनाओ।
> Same user को हमेशा same variant return करो।

import crypto from 'crypto';

interface Experiment {
  id: string;
  variants: { id: string; weight: number }[];
}

class ABTestEngine {
  private experiments: Map<string, Experiment> = new Map();

  register(experiment: Experiment) {
    const totalWeight = experiment.variants.reduce((sum, v) => sum + v.weight, 0);
    if (Math.abs(totalWeight - 100) > 0.01) {
      throw new Error('Variants के weights का total 100 होना चाहिए');
    }
    this.experiments.set(experiment.id, experiment);
  }

  assign(experimentId: string, userId: string): string {
    const experiment = this.experiments.get(experimentId);
    if (!experiment) throw new Error(`Experiment नहीं मिला: ${experimentId}`);

    const hash = crypto
      .createHash('md5')
      .update(`${experimentId}:${userId}`)
      .digest('hex');
    const bucket = parseInt(hash.substring(0, 8), 16) % 100;

    let cumulative = 0;
    for (const variant of experiment.variants) {
      cumulative += variant.weight;
      if (bucket < cumulative) return variant.id;
    }
    return experiment.variants[0].id;
  }
}

// Usage example
const engine = new ABTestEngine();
engine.register({
  id: 'checkout-flow',
  variants: [
    { id: 'control', weight: 50 },
    { id: 'new-design', weight: 50 },
  ],
});

React Component में उपयोग

import { createContext, useContext, useEffect } from 'react';

function useExperiment(experimentId: string): string {
  const engine = useContext(ABTestContext);
  const userId = useCurrentUserId();
  const variant = engine.assign(experimentId, userId);

  useEffect(() => {
    trackEvent('experiment_exposure', {
      experimentId,
      variant,
      userId,
    });
  }, [experimentId, variant, userId]);

  return variant;
}

// Component में उपयोग
function CheckoutPage() {
  const variant = useExperiment('checkout-flow');

  return variant === 'new-design'
    ? <NewCheckoutFlow />
    : <CurrentCheckoutFlow />;
}

Results की Statistical Analysis

A/B test के results को सही से evaluate करने के लिए, statistical significance की calculation ज़रूरी है।

interface TestResult {
  sampleSize: number;
  conversions: number;
}

function calculateSignificance(control: TestResult, treatment: TestResult) {
  const p1 = control.conversions / control.sampleSize;
  const p2 = treatment.conversions / treatment.sampleSize;

  const pooledP = (control.conversions + treatment.conversions) /
    (control.sampleSize + treatment.sampleSize);

  const se = Math.sqrt(
    pooledP * (1 - pooledP) * (1 / control.sampleSize + 1 / treatment.sampleSize)
  );

  const zScore = (p2 - p1) / se;
  const pValue = 2 * (1 - normalCDF(Math.abs(zScore)));

  return {
    controlRate: (p1 * 100).toFixed(2) + '%',
    treatmentRate: (p2 * 100).toFixed(2) + '%',
    improvement: (((p2 - p1) / p1) * 100).toFixed(2) + '%',
    pValue: pValue.toFixed(4),
    significant: pValue < 0.05,
  };
}

function normalCDF(x: number): number {
  const a1 = 0.254829592, a2 = -0.284496736;
  const a3 = 1.421413741, a4 = -1.453152027;
  const a5 = 1.061405429, p = 0.3275911;
  const sign = x < 0 ? -1 : 1;
  x = Math.abs(x) / Math.sqrt(2);
  const t = 1.0 / (1.0 + p * x);
  const y = 1.0 - ((((a5 * t + a4) * t + a3) * t + a2) * t + a1) * t * Math.exp(-x * x);
  return 0.5 * (1.0 + sign * y);
}

Summary

Claude Code का उपयोग करके, user allocation से लेकर statistical significance की calculation तक, A/B testing infrastructure को consistently बनाया जा सकता है। Feature flags के साथ integration के लिए Feature Flags Implementation देखें, और analytics integration के लिए Analytics Implementation Guide देखें।

Statistical testing की theory के लिए Evan Miller - Sample Size Calculator reference के रूप में उपयोगी है।