System Design Case Study: Designing a Scalable URL Shortener (like bit.ly)

1. Functional Requirements

Given a long URL → return a significantly shorter URL (e.g., https://short.ly/abc123)
When user visits the short URL → redirect (301/302) to the original long URL
Support custom aliases (optional)
Short URLs should expire after a configurable TTL (e.g., 1 year default)
High throughput: millions of new URLs per day
Very low latency on redirection (most critical path)

2. Non-Functional Requirements

High availability (99.99%+)
Extremely fast redirects (< 50ms p99)
Scalability: handle 100M+ short URLs and billions of redirects/month
Eventual consistency acceptable for analytics
Strong consistency not required

3. Estimation (Back-of-the-envelope)

New URLs/day: 10M → ~116 QPS write
Redirects/day: 1B → ~11,574 QPS read (read:write ≈ 100:1)
Storage: 10M URLs/year × ~500 bytes metadata → ~5 GB/year (tiny)
Hot redirects dominate traffic

4. High-Level Architecture

Key Insight: 99%+ of traffic is GET /:shortCode → 301 redirect → Optimize the redirect path aggressively.

5. Core Design Decisions

5.1 Short Code Generation Strategy

Option Chosen: Base62 Encoding of Auto-Incrementing ID (Best for scale)

Use a distributed 64-bit sequence generator (e.g., Snowflake ID or database AUTO_INCREMENT with sharding)
Convert the 64-bit integer to Base62 → 7–8 characters (a-z, A-Z, 0-9)
Example: ID = 1000000 → Base62 = “4c92”

Advantages:

No collision risk
Sequential IDs → predictable, cache-friendly
Easy to implement distributed counter

5.2 Database Choice

Primary DB: Apache Cassandra or AWS DynamoDB or Aerospike

Partition key: short_code
Columns: long_url, created_at, expires_at, user_id, custom_alias

Why NoSQL wide-column/partitioned store?

Redirect path becomes single-partition read → O(1) and extremely fast
Horizontal scaling built-in

5.3 Caching Strategy

Redis Cluster (multi-region if needed)
Cache mapping: short_code → long_url with TTL
Cache warming: preload top 1M most popular URLs
95–99% cache hit ratio expected on redirect path

5.4 API vs Redirect Separation

/api/v1/shorten → API servers (rate-limited, authenticated)
/:shortCode → Dedicated redirect service (no auth, minimal logic)

6. Detailed Data Flow

Create Short URL

Client sends long_url (and optional custom alias)
If custom alias → check uniqueness
Else generate next ID from distributed counter
Convert ID → Base62 short code
Insert into Cassandra: {short_code → long_url, expires_at…}
Invalidate/publish to Redis
Return https://short.ly/{short_code}

Redirect Flow (Critical Path)

User hits https://short.ly/abc123
CDN cache miss → Load balancer → Redirect service
Check Redis → HIT (99% case) → 302 → long_url
MISS → Query Cassandra by short_code → cache in Redis → redirect

7. C# .NET Core Implementation (Production-Ready Structure)

Project: UrlShortener.sln

src/
├── UrlShortener.Api/                  (ASP.NET Core Minimal API for shorten endpoint)
├── UrlShortener.Redirect/             (High-performance redirect service)
├── UrlShortener.Core/                 (Shared models, services)
├── UrlShortener.Infrastructure/       (Redis, Cassandra, ID generation)
└── UrlShortener.Worker/               (Background analytics, cleanup)

src/
├── UrlShortener.Api/                  (ASP.NET Core Minimal API for shorten endpoint)
├── UrlShortener.Redirect/             (High-performance redirect service)
├── UrlShortener.Core/                 (Shared models, services)
├── UrlShortener.Infrastructure/       (Redis, Cassandra, ID generation)
└── UrlShortener.Worker/               (Background analytics, cleanup)

7.1 Core Models (UrlShortener.Core)

public record ShortenRequest(string LongUrl, string? CustomAlias = null, DateTime? ExpiresAt = null);
public record ShortenResponse(string ShortUrl, string LongUrl, string ShortCode, DateTime ExpiresAt);

public class UrlMapping
{
    public string ShortCode { get; set; } = string.Empty;
    public string LongUrl { get; set; } = string.Empty;
    public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
    public DateTime? ExpiresAt { get; set; }
    public string? UserId { get; set; }
}

public record ShortenRequest(string LongUrl, string? CustomAlias = null, DateTime? ExpiresAt = null);
public record ShortenResponse(string ShortUrl, string LongUrl, string ShortCode, DateTime ExpiresAt);

public class UrlMapping
{
    public string ShortCode { get; set; } = string.Empty;
    public string LongUrl { get; set; } = string.Empty;
    public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
    public DateTime? ExpiresAt { get; set; }
    public string? UserId { get; set; }
}

7.2 Distributed ID Generator (Infrastructure/Services/DistributedIdGenerator.cs)

public interface IDistributedIdGenerator
{
    Task<long> NextIdAsync(CancellationToken ct = default);
}

// Using Cassandra-based counter (simple and reliable)
public class CassandraTicketIdGenerator : IDistributedIdGenerator
{
    private const string Key = "global_url_id";
    private readonly ISession _session;

    public CassandraTicketIdGenerator(ICluster cluster)
    {
        _session = cluster.Connect("url_shortener_keyspace");
    }

    public async Task<long> NextIdAsync(CancellationToken ct = default)
    {
        const string cql = @"
            UPDATE id_counter SET next_id = next_id + 1 WHERE key = ?;
            SELECT next_id FROM id_counter WHERE key = ?;";

        var stmt1 = new SimpleStatement(cql.Split(';')[0], Key);
        var stmt2 = new SimpleStatement(cql.Split(';')[1], Key);

        await _session.ExecuteAsync(stmt1);
        var row = await _session.ExecuteAsync(stmt2);
        return row.First().GetValue<long>("next_id");
    }
}

public interface IDistributedIdGenerator
{
    Task<long> NextIdAsync(CancellationToken ct = default);
}

// Using Cassandra-based counter (simple and reliable)
public class CassandraTicketIdGenerator : IDistributedIdGenerator
{
    private const string Key = "global_url_id";
    private readonly ISession _session;

    public CassandraTicketIdGenerator(ICluster cluster)
    {
        _session = cluster.Connect("url_shortener_keyspace");
    }

    public async Task<long> NextIdAsync(CancellationToken ct = default)
    {
        const string cql = @"
            UPDATE id_counter SET next_id = next_id + 1 WHERE key = ?;
            SELECT next_id FROM id_counter WHERE key = ?;";

        var stmt1 = new SimpleStatement(cql.Split(';')[0], Key);
        var stmt2 = new SimpleStatement(cql.Split(';')[1], Key);

        await _session.ExecuteAsync(stmt1);
        var row = await _session.ExecuteAsync(stmt2);
        return row.First().GetValue<long>("next_id");
    }
}

7.3 Base62 Encoder (Core/Services/Base62Encoder.cs)

public static class Base62Encoder
{
    private const string Characters = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";

    public static string Encode(long number)
    {
        if (number == 0) return "0";
        
        var sb = new StringBuilder();
        while (number > 0)
        {
            sb.Insert(0, Characters[(int)(number % 62)]);
            number /= 62;
        }
        return sb.ToString().PadLeft(7, '0'); // optional padding
    }

    public static long Decode(string str)
    {
        long result = 0;
        foreach (char c in str)
        {
            result = result * 62 + Characters.IndexOf(c);
        }
        return result;
    }
}

public static class Base62Encoder
{
    private const string Characters = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";

    public static string Encode(long number)
    {
        if (number == 0) return "0";
        
        var sb = new StringBuilder();
        while (number > 0)
        {
            sb.Insert(0, Characters[(int)(number % 62)]);
            number /= 62;
        }
        return sb.ToString().PadLeft(7, '0'); // optional padding
    }

    public static long Decode(string str)
    {
        long result = 0;
        foreach (char c in str)
        {
            result = result * 62 + Characters.IndexOf(c);
        }
        return result;
    }
}

7.4 Repository (Infrastructure/Repositories/UrlMappingRepository.cs)

public interface IUrlMappingRepository
{
    Task<UrlMapping?> GetByShortCodeAsync(string shortCode, CancellationToken ct);
    Task SaveAsync(UrlMapping mapping, CancellationToken ct);
}

public class CassandraUrlMappingRepository : IUrlMappingRepository
{
    private readonly ISession _session;
    private readonly PreparedStatement _insertPs;
    private readonly PreparedStatement _selectPs;

    public CassandraUrlMappingRepository(ICluster cluster)
    {
        _session = cluster.Connect("url_shortener_keyspace");

        _insertPs = _session.Prepare(
            "INSERT INTO url_mappings (short_code, long_url, created_at, expires_at) VALUES (?, ?, ?, ?)");

        _selectPs = _session.Prepare(
            "SELECT long_url, expires_at FROM url_mappings WHERE short_code = ?");
    }

    public async Task<UrlMapping?> GetByShortCodeAsync(string shortCode, CancellationToken ct)
    {
        var row = await _session.ExecuteAsync(_selectPs.Bind(shortCode));
        var first = row.FirstOrDefault();
        if (first == null) return null;

        var expiresAt = first.GetValue<DateTime?>("expires_at");
        if (expiresAt.HasValue && expiresAt.Value < DateTime.UtcNow)
            return null; // expired

        return new UrlMapping
        {
            ShortCode = shortCode,
            LongUrl = first.GetValue<string>("long_url"),
            ExpiresAt = expiresAt
        };
    }

    public async Task SaveAsync(UrlMapping mapping, CancellationToken ct)
    {
        var bound = _insertPs.Bind(
            mapping.ShortCode,
            mapping.LongUrl,
            mapping.CreatedAt,
            mapping.ExpiresAt);

        await _session.ExecuteAsync(bound);
    }
}

public interface IUrlMappingRepository
{
    Task<UrlMapping?> GetByShortCodeAsync(string shortCode, CancellationToken ct);
    Task SaveAsync(UrlMapping mapping, CancellationToken ct);
}

public class CassandraUrlMappingRepository : IUrlMappingRepository
{
    private readonly ISession _session;
    private readonly PreparedStatement _insertPs;
    private readonly PreparedStatement _selectPs;

    public CassandraUrlMappingRepository(ICluster cluster)
    {
        _session = cluster.Connect("url_shortener_keyspace");

        _insertPs = _session.Prepare(
            "INSERT INTO url_mappings (short_code, long_url, created_at, expires_at) VALUES (?, ?, ?, ?)");

        _selectPs = _session.Prepare(
            "SELECT long_url, expires_at FROM url_mappings WHERE short_code = ?");
    }

    public async Task<UrlMapping?> GetByShortCodeAsync(string shortCode, CancellationToken ct)
    {
        var row = await _session.ExecuteAsync(_selectPs.Bind(shortCode));
        var first = row.FirstOrDefault();
        if (first == null) return null;

        var expiresAt = first.GetValue<DateTime?>("expires_at");
        if (expiresAt.HasValue && expiresAt.Value < DateTime.UtcNow)
            return null; // expired

        return new UrlMapping
        {
            ShortCode = shortCode,
            LongUrl = first.GetValue<string>("long_url"),
            ExpiresAt = expiresAt
        };
    }

    public async Task SaveAsync(UrlMapping mapping, CancellationToken ct)
    {
        var bound = _insertPs.Bind(
            mapping.ShortCode,
            mapping.LongUrl,
            mapping.CreatedAt,
            mapping.ExpiresAt);

        await _session.ExecuteAsync(bound);
    }
}

7.5 Redirect Service (High Performance) – UrlShortener.Redirect

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

// Redis cache
var redis = ConnectionMultiplexer.Connect("redis-cluster:6379");
var cache = redis.GetDatabase();

app.MapGet("/{shortCode}", async (string shortCode, IUrlMappingRepository repo) =>
{
    // 1. Try cache
    var cached = await cache.StringGetAsync($"url:{shortCode}");
    if (cached.HasValue)
    {
        return Results.Redirect(cached, permanent: true);
    }

    // 2. DB lookup
    var mapping = await repo.GetByShortCodeAsync(shortCode, default);
    if (mapping == null || (mapping.ExpiresAt.HasValue && mapping.ExpiresAt < DateTime.UtcNow))
    {
        return Results.NotFound();
    }

    // 3. Cache for 24h
    await cache.StringSetAsync($"url:{shortCode}", mapping.LongUrl, TimeSpan.FromHours(24));

    return Results.Redirect(mapping.LongUrl, permanent: true);
});

app.Run();

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

// Redis cache
var redis = ConnectionMultiplexer.Connect("redis-cluster:6379");
var cache = redis.GetDatabase();

app.MapGet("/{shortCode}", async (string shortCode, IUrlMappingRepository repo) =>
{
    // 1. Try cache
    var cached = await cache.StringGetAsync($"url:{shortCode}");
    if (cached.HasValue)
    {
        return Results.Redirect(cached, permanent: true);
    }

    // 2. DB lookup
    var mapping = await repo.GetByShortCodeAsync(shortCode, default);
    if (mapping == null || (mapping.ExpiresAt.HasValue && mapping.ExpiresAt < DateTime.UtcNow))
    {
        return Results.NotFound();
    }

    // 3. Cache for 24h
    await cache.StringSetAsync($"url:{shortCode}", mapping.LongUrl, TimeSpan.FromHours(24));

    return Results.Redirect(mapping.LongUrl, permanent: true);
});

app.Run();

7.6 Shorten API Service – UrlShortener.Api

app.MapPost("/api/v1/shorten", async (
    ShortenRequest req,
    IDistributedIdGenerator idGen,
    IUrlMappingRepository repo,
    HttpContext ctx) =>
{
    string shortCode;

    if (!string.IsNullOrEmpty(req.CustomAlias))
    {
        shortCode = req.CustomAlias;
        var exists = await repo.GetByShortCodeAsync(shortCode, default);
        if (exists != null) return Results.Conflict("Custom alias taken");
    }
    else
    {
        var id = await idGen.NextIdAsync();
        shortCode = Base62Encoder.Encode(id);
    }

    var mapping = new UrlMapping
    {
        ShortCode = shortCode,
        LongUrl = req.LongUrl,
        ExpiresAt = req.ExpiresAt ?? DateTime.UtcNow.AddYears(1)
    };

    await repo.SaveAsync(mapping, default);

    var shortUrl = $"https://{ctx.Request.Host}/{shortCode}";
    return Results.Ok(new ShortenResponse(shortUrl, mapping.LongUrl, shortCode, mapping.ExpiresAt!.Value));
})
.RequireAuthorization(); // if user-specific

app.MapPost("/api/v1/shorten", async (
    ShortenRequest req,
    IDistributedIdGenerator idGen,
    IUrlMappingRepository repo,
    HttpContext ctx) =>
{
    string shortCode;

    if (!string.IsNullOrEmpty(req.CustomAlias))
    {
        shortCode = req.CustomAlias;
        var exists = await repo.GetByShortCodeAsync(shortCode, default);
        if (exists != null) return Results.Conflict("Custom alias taken");
    }
    else
    {
        var id = await idGen.NextIdAsync();
        shortCode = Base62Encoder.Encode(id);
    }

    var mapping = new UrlMapping
    {
        ShortCode = shortCode,
        LongUrl = req.LongUrl,
        ExpiresAt = req.ExpiresAt ?? DateTime.UtcNow.AddYears(1)
    };

    await repo.SaveAsync(mapping, default);

    var shortUrl = $"https://{ctx.Request.Host}/{shortCode}";
    return Results.Ok(new ShortenResponse(shortUrl, mapping.LongUrl, shortCode, mapping.ExpiresAt!.Value));
})
.RequireAuthorization(); // if user-specific

8. Scaling & Optimization Summary

Component	Technology	Scaling Strategy
Redirect Service	.NET 8 + AOT	1000+ instances behind CDN
Cache	Redis Cluster	Sharded, multi-AZ
DB	Cassandra/DynamoDB	Partitioned by short_code
ID Generation	Cassandra counter	Lightweight, linear scalable
CDN	Cloudflare/CloudFront	Caches 301 responses

9. Final Performance Expectation

p99 redirect latency: < 20ms (with CDN + Redis hit)
Write QPS: 10K+ sustainable
Storage cost: <$50/month at 1B URLs

This design powers real-world services like bit.ly, tinyurl, and rebrand.ly at global scale using similar principles. The C# .NET implementation above is production-grade and deployable on Kubernetes or AWS ECS/Fargate.