Currency rates¶
This tutorial deploys cbu-rates, a service that polls the exchange rates of the Central Bank of Uzbekistan (CBU)
every 60 seconds, keeps a history of every change, serves a small public API and notifies another service when a
rate moves. It is for developers who want to see schedules, outbound fetch, change detection and queue
notifications working together.
Deno
What you build¶
- A poller that runs on an interval schedule, stores a history row only when a rate or its date changes, and backs off when the CBU is down.
- A public read API: latest rates, history, conversion between any two currencies (UZS included) and the poller's status.
- Admin routes to poll now, ingest a payload by hand and backfill history from the CBU archive.
- Optional notifications: a
cbu.rate_changedmessage to another service's queue, for example the Telegram bot, which forwards it to its subscribers.
Why it fits the runtime. A poller is the textbook scheduled job: a short task every minute, nothing running in
between. The runtime fires the schedule, never runs two ticks of one schedule at once and skips missed ticks, so
the code does not need a lock. Each poll is one small outbound request, the history is a few SQL rows, and the
notification is one enqueue with a dedup id.
Architecture¶
flowchart LR
SCH["Interval schedule (60 s)"] -- "scheduled()" --> CBU["cbu-rates (Deno, Form B)"]
CBU -- "fetch (10 s timeout)" --> FEED["cbu.uz JSON feed"]
CBU --- KV[("std:kv: poll status")]
CBU --- SQL[("std:sql: latest, history")]
CBU -- "enqueue cbu.rate_changed (dedupId)" --> BOT["NOTIFY_SERVICE (e.g. telegram-bot)"]
U["Clients"] -- "GET /latest /history /convert /status" --> CBU
- The schedule calls
scheduled(). If the poller is backing off after failures, it returns at once. - It fetches
https://cbu.uz/uz/arkhiv-kursov-valyut/json/with a 10 s timeout and a 1 MB cap, and validates every item. - In one SQL transaction it compares the new rates with the stored latest ones, adds history rows and updates the latest values.
- If a rate moved for a watched currency, it enqueues one notification to
NOTIFY_SERVICE.
Prerequisites¶
- A running node and a tenant key in
RT_KEY(see First run). 21ci(see Install), orbash,curlandjqforsample/scripts/*.sh.- The node must be able to reach
cbu.uz(outboundfetchgoes to the public internet only).
| Name | Kind | Meaning |
|---|---|---|
ADMIN_TOKEN |
secret | Bearer token for /admin/* (at least 16 characters; not a tenant key, since the platform removes Authorization: Bearer rtk_…) |
NOTIFY_SERVICE |
env | Slug of a same-tenant service with a queue handler; unset = no notifications |
NOTIFY_CCYS |
env | Currencies to announce, default USD,EUR,RUB |
CBU_BASE_URL |
env | Feed base URL, default https://cbu.uz/uz/arkhiv-kursov-valyut/json/ |
Deploy it¶
1. Push the code and set the admin token¶
cd sample/apps/cbu-rates
RT_COMPILE=eager ../../scripts/deploy.sh src cbu-rates
SVC=$(../../scripts/svc-id.sh cbu-rates)
ADMIN=$(openssl rand -hex 24)
jq -nc --arg v "$ADMIN" '{value: $v}' | curl -sS -X PUT -H "Authorization: Bearer $RT_KEY" \
-H 'Content-Type: application/json' --data-binary @- "$RT_API/v1/services/$SVC/secrets/ADMIN_TOKEN"
2. Add the schedule¶
60 seconds is the shortest interval the free plan allows; a faster schedule is refused with
422 schedule_too_frequent. The feed changes about once a day, so --every 5m is just as good. See
Schedules.
3. Fill the history and connect a consumer (optional)¶
# 30 days of history from the CBU archive
curl -sS -X POST -H "Authorization: Bearer $ADMIN" 'http://127.0.0.1:8080/s/cbu-rates/admin/backfill?days=30'
# rate alerts through the Telegram bot (same tenant and environment)
21ci env set cbu-rates NOTIFY_SERVICE=telegram-bot NOTIFY_CCYS=USD,EUR,RUB
Try it¶
B=http://127.0.0.1:8080/s/cbu-rates
curl -sS "$B/latest?ccy=USD,EUR,RUB"
curl -sS "$B/history?ccy=USD&days=7" | jq '.points[] | [.date, .rate]'
curl -sS "$B/convert?amount=100&from=USD&to=EUR"
curl -sS "$B/status"
| Method and path | Auth | Answer |
|---|---|---|
GET /latest?ccy=USD,EUR |
none | {source, date, rates: [{ccy, code, name, nominal, rate, diff, date, perUnit}]}; 404 no_data before the first poll |
GET /history?ccy=USD&days=30 |
none | {ccy, days, points: [{date, rate, diff, nominal, perUnit, observedAt}]}, days 1-366 |
GET /convert?amount=100&from=USD&to=EUR |
none | {amount, from, to, result, rates}; to defaults to UZS |
GET /status |
none | {lastAttemptAt, lastSuccessAt, consecutiveFailures, lastError, backingOff, nextAttemptAt} |
POST /admin/refresh |
admin | poll now, ignoring the back-off: 200 {fetched, skipped, changed, notify} or 502 |
POST /admin/ingest |
admin | a CBU-format JSON array (≤ 1 MB), processed like a poll; 422 if no item is valid |
POST /admin/backfill?days=30 |
admin | the archive for each of the last days (≤ 31) days, 4 requests at a time |
Rates are decimal strings exactly as the CBU publishes them ("11778.45" UZS per nominal units); perUnit is a
number for convenience.
How the code works¶
src/main.ts holds the handlers and all I/O. src/cbu.ts is pure: feed parsing, change detection, the back-off
rule and conversion. src/http.ts has JSON, auth and a size-capped body reader.
Two entry points: fetch and scheduled¶
if (rates.length === 0) return error(422, "no_valid_rates", errors.slice(0, 10).join("; ") || "empty payload");
const { forward, older } = await ingest(rates, "ingest");
return json({
accepted: rates.length,
skipped: errors,
changed: forward.map((c) => ({ ccy: c.ccy, date: c.date, rate: c.rate, previous: c.previous })),
olderAdded: older,
notify: await notify(forward),
});
// … the public routes …
endpoints: ["/latest?ccy=USD,EUR", "/history?ccy=USD&days=30", "/convert?amount=100&from=USD&to=UZS", "/status"],
});
case "/latest":
return await handleLatest(url);
case "/history":
return await handleHistory(url);
case "/convert":
return await handleConvert(url);
case "/status":
return await handleStatus();
}
return error(404, "not_found", "no such route");
The interval schedule calls scheduled(event, ctx); HTTP requests go to fetch. A failed poll is recorded in KV,
not thrown: the next tick (or the back-off) is the retry. Scheduled tasks get a longer wall limit than HTTP ones
(300 s by default, capped by the plan's limits_max: 30 s on free), see Limits.
Fetching an outside API defensively¶
}
return { forward, older: added };
});
}
// ---------------------------------------------------------------------------------- feed
type FeedResult = { ok: true; rates: CbuRate[]; errors: string[] } | { ok: false; error: string; detail?: string[] };
async function fetchFeed(path: string): Promise<FeedResult> {
const ac = new AbortController();
const timer = setTimeout(() => ac.abort(), FETCH_TIMEOUT_MS);
try {
const res = await fetch(feedBase() + path, { headers: { accept: "application/json" }, signal: ac.signal });
if (res.status !== 200) {
await res.body?.cancel().catch(() => {});
return { ok: false, error: `upstream_http_${res.status}` };
}
const text = await readTextCapped(res.body, MAX_FEED_BYTES);
if (text === null) return { ok: false, error: "upstream_too_large" };
let raw: unknown;
try {
raw = JSON.parse(text);
} catch {
return { ok: false, error: "upstream_malformed" }; // an HTML maintenance page, a cut-off body, …
}
const { rates, errors } = parseCbuPayload(raw);
if (rates.length === 0) return { ok: false, error: "upstream_no_valid_rates", detail: errors.slice(0, 10) };
- An
AbortControllertimeout of 10 seconds keeps a hanging upstream from using the whole wall time. readTextCappedstops reading after 1 MB, so an oversized or endless body cannot exhaust memory.- Non-
200answers, HTML maintenance pages, cut-off bodies and empty arrays all count as failures with a stable error code. When the runtime refuses the request itself,fetchrejects with a code such asegress_timeoutoregress_dns, which ends up inlastError. - Single bad items are skipped and logged; the rest of the payload is kept.
Change detection in one transaction¶
id INTEGER PRIMARY KEY AUTOINCREMENT, ccy TEXT NOT NULL, date TEXT NOT NULL, rate TEXT NOT NULL,
diff TEXT NOT NULL, nominal INTEGER NOT NULL, observed_at INTEGER NOT NULL, source TEXT NOT NULL,
UNIQUE (ccy, date, rate))`,
`CREATE INDEX IF NOT EXISTS history_ccy_date ON history (ccy, date)`,
], "write");
}
schemaReady = true;
}
const INSERT_HISTORY = `INSERT OR IGNORE INTO history (ccy, date, rate, diff, nominal, observed_at, source)
VALUES (?, ?, ?, ?, ?, ?, ?)`;
const UPSERT_LATEST = `INSERT INTO latest (ccy, code, name, nominal, rate, diff, date, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (ccy) DO UPDATE SET code = excluded.code, name = excluded.name,
nominal = excluded.nominal, rate = excluded.rate, diff = excluded.diff,
date = excluded.date, updated_at = excluded.updated_at
WHERE excluded.date >= latest.date`;
// Compare and write in one transaction, so two concurrent polls cannot both record a change.
async function ingest(rates: CbuRate[], source: string): Promise<{ forward: Change[]; older: number }> {
await ensureSchema();
const now = Date.now();
return await sql.transaction(async (tx: SqlTx) => {
const latest = new Map<string, Latest>();
for (const row of objects(await tx.execute("SELECT ccy, date, rate FROM latest"))) {
latest.set(String(row.ccy), { ccy: String(row.ccy), date: String(row.date), rate: String(row.rate) });
}
const { forward, older } = detectChanges(latest, rates);
for (const r of forward) {
await tx.execute(INSERT_HISTORY, [r.ccy, r.date, r.rate, r.diff, r.nominal, now, source]);
The comparison with latest and the writes run in one sql.transaction, so a scheduled poll and an admin refresh
running at the same moment cannot both record the same change. History is idempotent by construction: one row per
(ccy, date, rate) through a unique key and INSERT OR IGNORE, and the upsert of latest only moves forward in
time (WHERE excluded.date >= latest.date).
The comparison itself is a pure function:
// Compares fetched rates with the stored latest ones.
// forward: a new currency, a newer date, or a corrected rate for the same date
// -> history row + new latest value
// older: a date before the stored one (backfill, a lagging mirror) -> history only
// unchanged rates produce nothing.
export function detectChanges(latest: Map<string, Latest>, incoming: CbuRate[]): { forward: Change[]; older: CbuRate[] } {
const forward: Change[] = [];
const older: CbuRate[] = [];
for (const r of incoming) {
const prev = latest.get(r.ccy);
if (!prev) {
forward.push({ ...r, previous: null, previousDate: null });
} else if (r.date < prev.date) {
older.push(r);
} else if (r.date !== prev.date || r.rate !== prev.rate) {
forward.push({ ...r, previous: prev.rate, previousDate: prev.date });
}
}
return { forward, older };
}
// Which changes users should hear about: a known currency whose rate actually moved.
export function notifiable(changes: Change[], ccys: string[]): Change[] {
return changes.filter((c) => c.previous !== null && c.rate !== c.previous && ccys.includes(c.ccy));
}
// Stable id of a change set, used as the queue dedupId (the same change is announced once).
export async function changeKey(changes: Change[]): Promise<string> {
const text = changes.map((c) => `${c.ccy}:${c.date}:${c.rate}`).sort().join("|");
const digest = new Uint8Array(await crypto.subtle.digest("SHA-256", new TextEncoder().encode(text)));
return `cbu:${Array.from(digest.slice(0, 16), (b) => b.toString(16).padStart(2, "0")).join("")}`;
}
A new currency, a newer date or a corrected rate for the same date is a forward change. An older date (backfill, a lagging mirror) only adds history. Notifications go out only when the rate itself moved for a watched currency, never for the first value seen.
Notifying another service¶
// ---------------------------------------------------------------------------------- notify
interface NotifyResult {
enqueued: boolean;
reason?: string;
key?: string;
count?: number;
}
// Optional: works without a consumer. NOTIFY_SERVICE is the slug of a same-tenant service with a
// queue handler (the telegram-bot sample). dedupId = hash of the changes: a retried poll or a
// concurrent admin refresh announces the same change once (24 h dedupe window).
async function notify(changes: Change[]): Promise<NotifyResult> {
const target = env("NOTIFY_SERVICE");
if (!target) return { enqueued: false, reason: "NOTIFY_SERVICE is not set" };
const ccys = parseCcyList(env("NOTIFY_CCYS")) ?? ["USD", "EUR", "RUB"];
const relevant = notifiable(changes, ccys);
if (relevant.length === 0) return { enqueued: false, reason: "no rate change in " + ccys.join(",") };
const key = await changeKey(relevant);
const body = {
type: "cbu.rate_changed",
key,
changes: relevant.map((c) => ({ ccy: c.ccy, date: c.date, rate: c.rate, diff: c.diff, nominal: c.nominal, previous: c.previous })),
};
try {
enqueue(target, body, {dedupId})puts a message on the queue of another service in the same tenant. The target must admit the caller: by default only services in the same project and environment may enqueue to it (see Calls and queues).dedupIdis a hash of the change set. The samededupIdwithin 24 hours is delivered once, so a retried poll or a concurrent refresh announces a change once.- Failures (
not_found,call_not_allowed,forbidden,queue_full) are logged and returned innotify.reason. They never fail the poll.
Status and back-off in KV¶
// not_found (no such service), call_not_allowed (other environment), forbidden (queue trigger off), queue_full
const code = (e as { code?: string }).code ?? "enqueue_failed";
console.warn(`notify ${target} failed: ${code}`);
return { enqueued: false, reason: code, key };
}
}
// ---------------------------------------------------------------------------------- poll
async function readStatus(): Promise<FetchStatus> {
const s = await kv.get(STATUS_KEY);
return s && typeof s === "object" ? s as FetchStatus : {};
}
async function poll(source: string) {
const status = await readStatus();
const now = Date.now();
const feed = await fetchFeed("");
if (!feed.ok) {
const failures = (status.consecutiveFailures ?? 0) + 1;
await kv.set(STATUS_KEY, { ...status, lastAttemptAt: now, consecutiveFailures: failures, lastError: feed.error });
console.warn(`poll failed (${failures} in a row): ${feed.error}`);
// Past the first failures the polls back off: one retry at the end of the back-off, as a
// one-off schedule (the same id replaces the pending one, so there is at most one).
const retryAt = nextAttemptAt({ consecutiveFailures: failures, lastAttemptAt: now });
if (retryAt > 0) {
await schedule.at(retryAt, { reason: "back-off", source }, { id: RETRY_SCHEDULE }).catch((e) =>
console.warn(`retry not scheduled: ${errorText(e)}`)
);
}
return { ok: false as const, error: feed.error, detail: feed.detail, consecutiveFailures: failures };
}
const { forward, older } = await ingest(feed.rates, source);
const notified = await notify(forward);
// Circuit breaker for the 60 s schedule: after 3 failures in a row, wait 1, 2, 4 … up to 30 min
// between attempts, so an outage at the CBU is not hammered every minute.
export function nextAttemptAt(s: FetchStatus): number {
const failures = s.consecutiveFailures ?? 0;
if (failures < 3 || !s.lastAttemptAt) return 0;
return s.lastAttemptAt + Math.min(60_000 * 2 ** (failures - 3), 30 * 60_000);
}
The poll status is one KV document. After three failures in a row, nextAttemptAt spaces the attempts out to 1, 2,
4 … minutes, capped at 30 minutes, so an outage at the CBU is not hit every minute. GET /status shows the state;
POST /admin/refresh ignores the back-off.
Backfill within the wall limit¶
lastAttemptAt: now,
lastSuccessAt: now,
consecutiveFailures: 0,
lastError: null,
lastChangeAt: forward.length ? now : (status as Record<string, unknown>).lastChangeAt ?? null,
});
await schedule.cancel(RETRY_SCHEDULE).catch(() => false);
if (forward.length) console.info(`changed: ${forward.map((c) => `${c.ccy} ${c.previous ?? "-"} -> ${c.rate} (${c.date})`).join(", ")}`);
return {
ok: true as const,
fetched: feed.rates.length,
skipped: feed.errors.length,
changed: forward.map((c) => ({ ccy: c.ccy, date: c.date, rate: c.rate, previous: c.previous })),
olderAdded: older,
notify: notified,
};
}
// The CBU archive, newest first, 4 requests at a time, stopping before the wall limit.
async function backfill(days: number, ctx: Ctx) {
const dates = Array.from({ length: days }, (_, i) => isoDate(Date.now(), i + 1));
Four workers share one list of dates, so at most four requests run at once. Each worker stops when less than 8
seconds of the task's wall time (ctx.deadline) remain, and the answer reports how many days were done. The
runtime also caps outbound requests per task (50 for an HTTP task by default, at most 6 at once), which a 31-day
backfill stays under.
Cacheable public reads¶
rows: unknown[][];
rowsAffected: number;
}
interface SqlTx {
execute(sql: string, args?: unknown[]): Promise<SqlResult>;
code: String(o.code),
name: String(o.name),
nominal,
rate: String(o.rate),
diff: String(o.diff),
date: String(o.date),
perUnit: perUnit(String(o.rate), nominal),
};
}
The read endpoints send cache-control: public, max-age=60 and access-control-allow-origin: *, so a CDN or a
browser page on any origin can use them directly.
Test it¶
Unit tests. test/cbu.test.ts covers decimal normalization, CBU date parsing, payload parsing that keeps good
items and reports bad ones, change detection (new, unchanged, newer date, correction, older), which changes are
notifiable, the stable change key, the back-off schedule and the capped body reader.
Smoke test. smoke.sh deploys cbu-rates-smoke. The deterministic part ingests synthetic CBU payloads dated 20
days ahead through /admin/ingest: a bad item is skipped, the same data twice stores nothing, a moved USD rate is a
change with its previous value, and a notification to a missing service fails softly. Then it checks /latest
(including a currency with nominal 10 and the CORS header), /history and /convert. The live part polls cbu.uz
once (a 502 with an upstream_* code during a CBU outage also passes).
RT_API=http://127.0.0.1:8081 RT_INVOKE=http://127.0.0.1:8080 RT_KEY=rtk_… ./smoke.sh
NOTIFY_SLUG=telegram-bot-smoke ./smoke.sh # notify a real consumer instead of a missing one
SMOKE_SLOW=1 ./smoke.sh # also adds a 60 s schedule and waits for one scheduled poll
Notes and limits¶
- Dates are calendar dates in Tashkent (UTC+5), which is how the CBU publishes them.
/historyaccepts up to 366 days,/admin/backfillup to 31 days per call.- Polling every 60 seconds suits a feed that changes once a day only because each poll is cheap (one request, about 26 KB).
Ideas to extend it¶
- Watch a second source (a commercial bank's rates) with another schedule and compare spreads.
- Add a
cronschedule for a daily digest (cron schedules need a plan that allows them; thefreeplan does not). - Serve a small chart page from the service, reading
/history. - Fan notifications out to several consumers by enqueueing to more than one slug.
Where the code lives¶
sample/apps/cbu-rates/ in the runtime repository: src/ (the service), test/ (unit tests), smoke.sh and a
README.
See also¶
- Schedules: interval and cron schedules, overlap and skipped ticks
- Calls and queues:
enqueue, dedupe, environments and allowed callers - Telegram bot: the consumer of the notifications
21ci schedule