Segun Sogunle

    essay

    What Happens When the Internet Goes Off?

    Availability, designed from a place where the power grid is part of your threat model. Reliability lessons the textbooks cannot teach.

    In 2020 I was designing a customer service platform for the Nigerian market, and in my planning notes, under a heading called Concerns, I wrote a question I have never seen in a Western architecture document:

    What happens when the internet goes off?

    And beneath it, my own answer at the time: what happens when there is no network on your phone? Meaning: this is not a hypothetical to be engineered away. Your users already live this daily, and they already have expectations about how software should behave through it. Six years later I still think that exchange, a question and its reframe, teaches more about availability than most of the literature, and this essay is my attempt to unpack it.

    Where the threat model actually starts

    The standard reliability stack begins at the service: replicas, zones, failover. Building for Lagos, the stack begins two layers lower. The startup budget I drew up that year had line items no Silicon Valley deck carries: inverter, batteries, solar. Not as sustainability garnish. As the power layer of the architecture, because the grid delivers electricity in intervals, and everything above it inherits that rhythm. Mobile data follows its own weather: a tower's backup power dies with the neighbourhood's; a backhaul link degrades at peak; data bundles simply run out, which is a failure mode of the user's wallet, not of any network element, and your system cannot tell the difference.

    Formally: connectivity is not a boolean. It is a probability distribution with time of day, geography, and income as parameters. The engineering follows directly once you accept that, and refuses to follow while you do not.

    Offline first sync topology The device holds the source of truth in a local store and an outbox queue. The queue drains to an idempotent, resumable sync endpoint whenever data is available. An SMS or USSD path reaches the same endpoint from a feature phone with no data plan. The endpoint feeds a reconciliation step that compares server truth with device truth. User’s world · intermittent by default 01 App: local store is the source of truth every write lands here first 02 Outbox queue every action durable, idempotency key attached data available: drain, any order, any time degrade to 03 SMS / USSD path works on a feature phone, no data plan required 04 Sync endpoint idempotent, resumable, redelivery is a no-op 05 Reconciliation server truth against device truth, conflicts resolved by rule, gaps detected by count
    Fig. 1 · The local store is primary; the network is an opportunistic replication channel.

    The design consequences, concretely

    The local store is the application. Offline-first is often presented as caching with ambition. It is the reverse: the device's store is primary, and the network is an opportunistic replication channel. Every user action lands locally, durably, immediately, and a queue drains whenever the distribution smiles. The user experiences software that always works; the distributed system underneath is doing eventual consistency without apology.

    Idempotency is not optional. A queue that drains over a flapping link will deliver duplicates; that is its job. Every operation therefore carries a key, and the server treats redelivery as a no-op. Engineers who learn distributed systems from textbooks meet idempotency as a nicety of API design. Engineers who learn it from a 2G cell in traffic meet it as the difference between a working product and double-charged customers.

    The lowest layer is not HTTP. My first shipped system, an electronic voting platform I built as an undergraduate, authenticated voters over SMS, because SMS was the channel every participant reliably had. That instinct never left. USSD and SMS are slow, tiny, and nearly universal: a payments or complaints flow that degrades to them keeps functioning on a feature phone with no data plan at all. Degradation paths are not a lesser experience bolted on for the poor; they are the system's honesty about the distribution it actually serves.

    Reconciliation replaces assumption. When device and server can disagree for hours, "did everything arrive?" becomes a first-class operational question. Counts compared across the boundary, windowed by time; gaps surfaced, not assumed absent. Readers of my observability writing will recognise the shape: a partitioned system that keeps accepting work quietly is precisely the kind that fails silently, so verification against intent stops being a luxury and becomes the availability mechanism itself.

    The part the textbooks get backwards

    Here is the reframe that matters. CAP, in the standard telling, is a theorem about an exotic condition: if a partition occurs, choose availability or consistency. In most of the world, the partition is not the exotic case. It is Tuesday. The choice the theorem describes is not made once in a design review; it is made continuously, by every queue depth and retry policy, and the systems that thrive are the ones that made it deliberately.

    Which is why I have come to see this constraint environment as a teacher rather than a handicap. Consider what a multi-region cloud architecture does during a regional outage: it queues writes, serves stale-but-useful reads, degrades noncritical features, reconciles when the region returns. That is precisely an offline-first mobile app, scaled up and given a marketing budget. The patterns that Western infrastructure celebrates as advanced resilience engineering, store and forward, idempotent replay, conflict resolution by rule, graceful degradation, are the entry-level requirements of building for intermittent environments. Engineers from these environments are not catching up to distributed systems practice. They grew up inside its hardest exam.

    There is a hardware chapter of the same lesson. Power that vanishes mid-write is a disk-corruption generator: it teaches you, personally and repeatedly, why fsync semantics matter, why write-ahead logs exist, why "the database was shut down uncleanly" is a sentence with consequences. Concepts that are abstractions in a reliable-grid country are muscle memory in an unreliable one.

    Where this leaves us

    The question from my 2020 notes was never really about the internet. It was about whether a system's model of the world includes the world's actual behaviour. A design that assumes connectivity, power, and payment are steady states will be fragile everywhere, and merely luckier in some geographies than others. A design that treats them as distributions produces the properties everyone claims to want: systems that degrade instead of failing, recover instead of forgetting, and verify instead of assuming.

    The internet goes off everywhere, eventually. Some of us just got to practise.

    Responses

    Threads run on GitHub Discussions. A GitHub account is needed to post. Corrections and counter examples are welcome, particularly from anyone who has run this in production.

    Discuss on GitHubReply by email

    ← All writing