Rebuilding the Bridge Between Your Keychain and the Open Financial Internet

1

Hi folks — it’s been a dense few weeks on the WebUI side of GRIDNET OS, and the work we’ve just landed rewrites one of the more subtle parts of the system: how your local identity talks to external blockchain networks and overlay services like Hyperliquid. If you’ve ever logged in, clicked around the Exchange dApp, or watched a QR code appear in the retro terminal and wondered “where exactly is all this stuff living?” — this update is for you.

The Problem We Started With

GRIDNET OS already has a beautiful primitive: CKeyChain. One multi-dimensional master key, fanning out into as many sub-identities as you want, each one its own isolated on-chain persona. Your GRIDNET native key, your Ethereum child key, all derived deterministically from the keychain and protected behind a single password unlock. Clean.

The problem appeared as soon as we started plugging in external services. Hyperliquid, the high-performance on-chain perp exchange, doesn’t care about your GRIDNET identity at all — it wants an Ethereum master address and an agent key (a short-lived secondary signing key that the master has pre-authorized to place orders on its behalf). That agent key has to persist across sessions, or the user faces the infamous “Extra agent already used” error every time they reload the page.

We’d hacked this together with a blob inside CAppSettings called hlApproval, and a supposed-to-be-deterministic agent key derived from the keychain via getChildEth(0x484C). Two things were wrong with that:

  1. The “per-sub-identity” claim was a lie. When we looked closely, the derivation was actually keychain-wide, not sub-identity-wide. All sub-identities of the same keychain shared the same Hyperliquid agent address. Nobody noticed because most users only ever used one sub-identity.

  2. It completely fell over for three important use-cases: users who wanted to import a raw Ethereum private key (not derived from the keychain), users connecting via Rabby / WalletConnect / Ledger (where we never get to see the master private key at all, only a signature), and users who just wanted to say “no Hyperliquid on this sub-identity, thanks.”

So we decided to do this properly.

The Big Idea: A Generic Vault Inside Your Keychain

What if any CKeyChain sub-identity could carry an arbitrary number of external identity records — one per blockchain network, plus one per overlay service that uses them — and the whole thing was encrypted at rest under the same ChaCha20 envelope that already protects your master key? No new storage layer. No new password. Just “if you can unlock your keychain, you can access everything attached to it; if you can’t, you can’t read any of it.”

That’s what we built. Two new stores live inside every CKeyChain instance now:

  • Network identities: per-sub-identity records describing which blockchains this sub-identity can act on (Ethereum today, Solana tomorrow), and where each one’s key material comes from. Three possible sources:

    • native — derived on demand from the keychain master (no storage, computed at read time)

    • imported — a raw private key the user pasted in, stored encrypted inside the vault

    • external — a Rabby / Ledger / WalletConnect wallet whose private key we will never see (we just record the address and the provider type)

  • Service associations: per-sub-identity records for overlay services like Hyperliquid, binding to one of the network identities and carrying the service’s own metadata — in HL’s case, the random ephemeral agent private key, the master address, the approval receipt, the 180-day expiry timestamp, and a fingerprint of the keychain it belongs to so it can’t be “moved” to a different keychain by sleight of hand.

Because everything sits inside the existing keychain blob, we got four important properties for free:

  • Encrypted at rest: same protection as your master key

  • Password-gated reads AND writes: nothing reads or writes the vault without an unlocked keychain

  • 5-minute auto-lock: the existing idle lock sweeps the vault access along with everything else

  • Per-sub-identity, per-keychain isolation: different sub-identities of the same keychain have fully independent records; different keychains entirely cannot see each other’s data

The vault API is entirely dApp-agnostic. Exchange is just the first consumer. Any future dApp — a Solana DEX, a Bitcoin LN gateway, a Uniswap permit manager — can store its own secrets by picking a namespace string and calling the same setServiceAssociationA() method. No changes to KeyChain or CKeyChainManager required.

The Hyperliquid Agent Key Is Now Truly Random

Here’s the subtle bit that took us the longest to get right. Hyperliquid’s approveAgent action binds a master Ethereum address to an agent Ethereum address. Nothing says the agent has to be “derived” from anything — it can be a random 32-byte secret as long as the master signs the approval. Deriving it deterministically from the keychain was an optimization we paid for with complexity: it only worked when the master itself was also keychain-derived, and it silently broke per-sub-identity isolation.

So we dropped it. The HL agent key is now generated fresh via crypto.getRandomValues(32) at the moment of first approval, and persisted in the vault. Every sub-identity gets its own agent. Every re-approval rotates the agent. An imported master, a Rabby master, or a native-derived master all behave identically — we generate the agent, the master signs the approval, we store the approval receipt plus the agent private key, and from that moment on trading signs locally from the cached agent.

The master is only needed again for operations that require the master’s signature: depositing funds from Arbitrum to HyperCore, withdrawing funds off the exchange, revoking the agent. Everything in between — place order, cancel, modify, TWAP, reduce-only, balance fetch, orderbook — runs from the in-memory agent cache with zero wallet popups. Rabby stays quiet; Ledger stays in the drawer.

The Epoch 2 Dialog

Once the vault started working, we noticed something: on re-login, we were just silently proceeding with whatever HL identity had been stored last time. Users had no chance to say “actually, I want to link a different wallet” or “disable HL on this sub-identity entirely” without digging into Exchange.

So we added a confirmation dialog that fires on every subsequent login where a valid HL record is found:

HYPERLIQUID ASSOCIATION FOUND

  Master:  0x3488…Fa05
  Agent:   0xabcd…1234
  Source:  Rabby Wallet
  Network: Hyperliquid Mainnet
  Approved: valid for 165 more days

  [1] Proceed with this association  (default, auto-selected in 5s)
  [2] Associate a different Hyperliquid identity
  [3] Disable Hyperliquid for this sub-identity

The default auto-proceeds after 5 seconds so nobody who just wants to keep trading is slowed down. Option 3 writes a “disabled” marker — a tiny record with just { v:2, disabled:true, subIndex, network } — that tells the boot flow this sub-identity has explicitly said no thanks, and next login it skips both the confirmation dialog AND the first-time setup prompt. Users re-enable HL from Exchange’s own “Enable One-Click Trading” banner when they’re ready.

Terminal Mode Is a First-Class Citizen Again

Here’s something I think many of you will appreciate. GRIDNET OS has a beautiful retro-terminal boot mode — amber CRT glow, green-phosphor alternative, scanlines, the works. It runs on xterm.js and it’s more than decoration: it’s a genuine alternative path into the OS for people who want a tactile, text-first experience.

The login flow, the HL wallet picker, the WalletConnect QR code, the error dialogs — all of them had to render meaningfully in both GUI mode (SweetAlert2 popups) and terminal mode (ANSI text + selection menus). We had a few places where the terminal path was an afterthought, rendering nothing, or rendering something unscannable. Three specific issues got run to ground:

1. The QR code that wouldn’t scan

When you picked [1] QR Code — Scan with mobile app on the terminal login screen, the mobile app QR code was rendered via DOM injection into document.body by the shared CQRIntent.show() method. In fullscreen xterm, that DOM is invisible — you saw “SELECTED” and then nothing. We wired requestQRLogon to detect terminal mode and route through our half-block ANSI renderer instead, using the exact same base58Check-encoded payload the GUI path uses so the mobile app receives byte-identical data. Added a 120-second expiry and any-key cancel so nobody gets stuck waiting for a scan that’s not coming.

2. The QR code with no finder patterns

Once we started rendering QR codes in the terminal, a second and more embarrassing bug surfaced: scanners couldn’t recognize them. Looking at the output, the three corner finder patterns — those square targets that let a scanner locate and align a QR code — were missing entirely. The payload bits were there but scrambled.

The root cause was wonderfully silly: our grid-building code sampled the library’s rendered canvas at a hard-coded pitch of floor(200 / 33) = 6 pixels, assuming every QR code was a 33-module version 3. But long URIs (WalletConnect’s URIs are hundreds of characters) force the library to pick a higher QR version — 45, 57, 67 modules wide — where each module is only 3-4 pixels. We were sampling at the wrong pitch, skipping entire rows, and producing output that looked QR-ish but had no structural anchors at all.

The fix was to bypass pixel-sampling entirely and read the module matrix directly from the QRCode library’s internal model via _oQRCode.isDark(row, col) + getModuleCount(). That’s the same authoritative data the library uses to draw its own canvas, so there’s zero aliasing risk regardless of version. We also kept a last-resort canvas-scan fallback that detects the actual module size from the top-left finder pattern’s 7-module width, in case we’re dealing with a library variant that doesn’t expose the internal model.

3. The aspect ratio fight

Terminal cells are taller than they are wide (Courier New is roughly 1:2). QR modules need to be square. We bounced between half-block rendering and full-block rendering a couple of times before settling on the right math: 1 QR column per terminal cell width, 2 QR rows per terminal row via Unicode half-block glyphs (▀, ▄, █, space). For a 1:2 cell, that’s 1 : 1 per module — perfectly square. Forced ESC[30;47m black-on-white ANSI overrides the CRT color theme (scanners need dark-on-light; amber-on-black is too unreliable). 4-module white quiet zone around the matrix per the QR spec.

Liveness: The User Always Lands Somewhere

The terminal login flow has a single unbreakable rule: the user must always reach the command-line prompt eventually. Even if the network is down, even if Hyperliquid’s API is unreachable, even if Rabby is hanging on a signature, even if the user walks away to make coffee — the flow must terminate in a reachable state, never hang forever.

We did a function-by-function audit and identified every await that could potentially block indefinitely. Then we added timeouts or guaranteed-resolution paths to each one:

  • Vault read: 15-second timeout, falls through to “no record” on expiry

  • HL API approveAgent submit: 30-second AbortController

  • WalletConnect session approval: 120-second race

  • Signature requests: 180-second race (hardware wallets need breathing room)

  • Password prompt: 120-second hard timeout

  • Source picker dialog: 60-second timeout, auto-defaults to “native”

  • Global HL flow ceiling: 5 minutes around the entire IIFE as a last-resort stopgap

Plus a bounded retry loop: if a per-source connect fails (Rabby rejected, WC timed out, Ledger locked), the user goes back to the source picker up to 3 times before the flow gives up and drops them cleanly into the terminal prompt. No infinite retry loops, no dead-ends.

On any ceiling timeout, we sweep every inflight dialog via cancelLocalUIRequest so no ghost dialogs block the final _promptViewingMode step. The command-line prompt always appears.

Passwords: You Get Three Tries, Not One

A subtler liveness issue: what if you mistype your keychain unlock password at login?

Before this refactor, the answer was “you get exactly one chance, and if you blow it, you’re dead-ended.” The unlock() method caught its own “Incorrect password” error and swallowed it into a generic return false, which the caller rendered as a bland “unlock failed” dialog. No re-prompt. No retry. You had to click the eclipse orb, walk through the login method picker, select your keychain again, and re-enter the password. The underlying 5-strike CKeyChainManager lockout counter was silently ticking up the whole time — you could get locked out for 5 minutes without ever being warned.

Now there’s an in-flow retry loop: three chances in a row to get the password right, each with an immediate re-prompt and a dual-counter error message showing both the in-session counter (“attempt 2 of 3”) and the underlying hard lockout counter (“3 attempts remaining before lockout”). Cancel and timeout return to the halo without penalty. Both terminal and GUI behave identically because the loop sits above the shared password dialog pipeline.

Explicitly at the user’s request: three is the max. We don’t spam more than that.

Logging: High-Fidelity, No Secrets

The whole flow now emits structured log events through the existing GRIDNET logging facility — console + UI log panel + mTools.logEvent persistent stream. Every step you might want to reconstruct later is logged: login start, vault read result, user choices in the confirmation dialog, wallet source selection, connection attempts, chain switches, signing requests, HL API responses, vault writes, auto-logon banner, errors.

A strict rule applies: no private key material, no raw signatures, no imported key bytes. Addresses are always truncated 6-and-4. Every log line has a timestamp and a level (info / warn / error / debug). 31 of 32 old console.log('[HL] ...') sites got migrated over; the one exception is a resource-loader callback where this isn’t bound to VMContext, and we left it alone with a comment.

Exchange dApp: Thinner, Faster, Cleaner

Exchange is now a pure consumer of CKeyChain services. No parallel key storage. No derivation paths. No imported-wallet persistence of its own.

AgentKeyManager got rewritten from scratch as a thin wrapper: loadFromVault(vmCtx) to populate from the freshly-read record, ensureAgentLoaded(vmCtx) to check TTL and re-read if stale, revokeAgent(vmCtx) to clear both the in-memory cache and the vault record on explicit disconnect, and clearAgent() to wipe the cache on logout. Everything else — deriveFromKeyChain, generateKey, approveAgent, saveApprovalStatus, _loadImportedWallet, _storePrivateKeyInSettings — is gone.

Two important UX decisions got baked in:

  1. The idle lock does not wipe the Exchange agent cache. When the 5-minute CKeyChainManager auto-lock fires, Exchange’s cached agent private key survives (it’s a detached Uint8Array, not a reference into the keychain). Trading continues uninterrupted. Only an explicit logout or window close clears it. This matches how the Wallet dApp already treats its own signing material, and it means you can walk away from your desk for 10 minutes and still place an order on your return without a password prompt mid-trade.

  2. A user-configurable cache TTL is now exposed in Exchange’s settings (right-click the chart → “Exchange Settings…”). Options: Never expire (default) / 15 minutes / 1 hour / 4 hours / 24 hours. If you set a non-zero TTL, the next order after expiry will force a vault re-read — which may require your keychain password if the manager has auto-locked in the meantime. Useful for shared machines; off by default for personal setups.

Plus a nice invisible touch: every time Exchange successfully signs an L1 action, it now calls CKeyChainManager.touch(handle) — a synchronous no-op idle-timer ping — so other dApps (Wallet, Settings) don’t get auto-locked under the user’s feet while you’re actively trading.

Atomic Password Changes Across Two Encryption Layers

One piece of infrastructure work that doesn’t get as much attention as it deserves: what happens when you change your keychain password? Before this refactor, changePassword() iterated every stored keychain, re-encrypted the main blob under the new password hash, and saved. It worked. Mostly.

Now that the vault sidecar exists alongside each main blob, we needed true atomicity across both layers for every keychain. If anything fails midway, we must not leave the user with some keychains on the new password and some on the old, or worse, main blobs migrated but sidecars orphaned.

We rewrote changePassword() as a strict five-phase commit:

  1. STAGE — decrypt every main blob and sidecar under the old hash, re-encrypt under the new hash, hold everything in memory. No storage writes yet.

  2. WRITE — setItem every new-hash entry to localStorage, tracking what we wrote so we can roll back.

  3. VERIFY — read every new-hash entry back, decrypt with the new hash, parse the sidecar JSON. Any failure triggers rollback.

  4. CLEANUP — only now delete the old-hash entries.

  5. MEMORY — update the current process’s unlock state to the new hash, invalidate other processes’ unlock states so they force re-unlock, reset the idle timer.

If anything fails in phases 1-3, we remove every entry we wrote in phase 2 and leave the user exactly in the pre-change state. Pending vault writes from other tabs are drained before phase 1 starts. Same-password no-ops short-circuit without touching storage. Keychain-list corruption can’t happen because the list itself is written and verified as part of the atomic transaction.

What’s Under the Hood That We Didn’t Touch

The whole refactor is additive on top of existing foundations. We did not touch:

  • The CKeyChain binary serialization format (getPackedData / unpack) — external tools that read stored keychains continue to work unchanged

  • The Wallet dApp (wallet.js) — verified end-to-end that all wallet flows are unaffected

  • The underlying ChaCha20 + SHA256(password) encryption envelope — same protection, same idle-lock semantics

  • Any existing dApp integration except Exchange

No migration prompts. No “please re-create your keychain” messages. Users with existing HL associations simply re-approve on next login (one-time cost of having a fresh 32-byte agent key), and everything else continues as before.

By the Numbers

  • 2 new in-memory stores added to CKeyChain

  • 12 new public CKeyChain methods

  • 13 new async CKeyChainManager methods

  • Encrypted sidecar per keychain, auto-loaded on getKeyChain()

  • 3 listener APIs with filter objects and disposer returns

  • 5-phase atomic password change across main blob + sidecar

  • 31 of 32 HL-flow log sites migrated to structured logging

  • 212 lines of legacy dead code deleted

  • 2 new test files covering vault roundtrip, persistence, concurrent writes, lock/change listeners, password migration

  • 20 liveness scenarios audited end-to-end and confirmed terminating

  • Every edited .js file passes node --check

What’s Next

A few things are in the backlog for follow-up passes:

  • Native Solana support — adding 'solana' to getNativeNetworks() via a getChildSol() child-key derivation method. The vault API is already set up for it; only the deriver needs wiring.

  • Per-namespace ACLs — currently any dApp with a valid processHandle can read any service association. This matches existing CKeyChain behavior (any dApp can read any child key), but a more granular permission model is on the table.

  • AEAD upgrade for CKeyChainManager’s on-disk encryption — current ChaCha20 lacks a MAC, so wrong passwords silently decrypt to garbage. Adding Poly1305 authentication would let us detect tampering and wrong passwords cryptographically.

  • Native local signing for deposits and withdrawals — currently those always pop the master wallet; for imported and native master sources we could sign locally the same way we do approveAgent.

The foundations are now in place to do all of these as additive, isolated changes. No more tangled ad-hoc persistence paths to untangle first.

Closing Thought

Security-sensitive refactors like this one are the kind of work that’s mostly invisible when it goes well. If we’ve done our job, nobody’s trading flow gets interrupted, nobody loses access to their HL positions, nobody notices anything except maybe that re-login to Hyperliquid now happens instantly and silently instead of walking them through a wallet popup. The visible payoff is small. The invisible payoff is a much cleaner foundation for every external-service integration GRIDNET OS will do from here on.

Thanks for following along. As always, issues, questions, and terminal-mode aesthetic feedback are welcome — the retro CRT glow deserves to be shown off.

— the GRIDNET OS DUI Team