You speak — a beat later the other person hears it in their language, as text and voice. Then they answer, and you hear them in yours. Six languages — English and five Indian ones — any pair, both ways, about a second and a half per turn, strictly one-to-one.
Email → one-time PIN · works on a phone · speech isn't recorded, sessions aren't stored
A clinic desk, a shop counter, a site office, a video call between families — the moment two people don't share a language, the conversation stalls into gestures, a third person translating, or an app you pass back and forth one sentence at a time.
Passing a phone back and forth, tapping, waiting — it turns a conversation into a transaction. Nobody looks up.
A translated sentence on a screen has no tone. In a real exchange you want to hear the other person answer, not read a caption.
Big translators are one-person, one-direction. A conversation is two people, both directions, at once — a different problem.
Setu is a room two people join from any browser. Each says which language they speak and which they want to hear. From then on you just talk — Setu listens, translates, and speaks it to the other person, both ways, while showing the text for both of you.
Talk naturally, or type a line / tap a quick phrase — both are translated and spoken to your partner.
A partner card shows Online · Speaking · Typing · Muted · Reconnecting · Left, so you always know what's happening on the other end.
Turn on video when you want to — camera off by default, the other person accepts or declines. Faces go browser-to-browser while the translated audio keeps flowing.
Echo cancellation and noise handling keep the other voice out of your mic; push-to-talk for loud places; a headphone tip when it matters.
Drops reconnect on their own and keep your transcript and call timer. Replay a line, delete for both, download the whole conversation.
Languages: English, Hindi, Bengali, Tamil, Telugu and Marathi — any pair, both directions. (A non-English pair like Hindi↔Telugu is translated through English behind the scenes, so it's a touch slower.) Turn latency was measured on the beta bridge; speech is turned to text, used once, and dropped.
The page and access gate are static and cloud-hosted. The translation itself runs on a bridge: for each person who speaks, audio flows through voice detection, speech-to-text, translation, and text-to-speech, then arrives at the other person as text and voice.
Every row states its own evidence. Where something is roadmap, network-dependent, or deliberately excluded, it says so — a matrix showing everything complete is a matrix nobody believes.
| Requirement | What was built | Evidence | Status |
|---|---|---|---|
| Real-time two-person voice translation | A room two people join; each speaks and hears their own language. Peer delivery
bypasses the busy path so the listener isn't blocked by their own pipeline.
Strictly one-to-one — no group calls by design. |
live on setu.smritiraj.online | Built · live |
| Six languages, any pair, both ways | English, Hindi, Bengali, Tamil, Telugu, Marathi — any pair, either way, as text and spoken voice (Tamil included). Non-English pairs route through English internally (a beat slower). | all six voiced by Piper | Built · live |
| Fast enough to feel like a conversation | Streaming speech-to-text, end-of-speech at 300 ms, translation beam trimmed 5→2, translate + voice moved off the pipeline thread. | ~1.5 s/turn on the bridge | Built · live |
| Type, quick phrases, and message actions | Type a line or tap a quick phrase (translated + spoken); per-message replay, copy, show-original, rate, and delete-for-both with undo. | P0/P1 shipped | Built · live |
| Know what the other person is doing | Partner card — Online / Speaking / Typing / Muted / Reconnecting / Left — plus a reconnect grace window and a language-change notice. | Phase C, two-account walk | Built · live |
| Device control, captions, notifications, help | Mic/speaker picker, large live caption, background-tab chime + unread title, keyboard shortcuts, text-size A−/A+, and an “Audio problem?” quick-fix panel. | Phase D, headless walk | Built · live |
| A complete way to end a call | Leave confirmation, a “partner left — wait or leave” choice, and an end screen with call duration, download-transcript, rejoin and start-new-call. | Phase E · 26 tests + headless walk | Built · live |
| Accessible to keyboard and screen-reader users | Focus-trapped dialogs with focus restore, a visible focus ring, AA-contrast greys, an aria-live region announcing partner-status changes, never colour alone. | Phase E · WCAG 2.1 AA pass | Built · live |
| See each other on video (optional) | An opt-in, camera-off-by-default peer-to-peer video layer: one person
starts video, the other accepts or declines, then you see each other's live
camera while the translated audio keeps flowing. Video is browser-to-browser (no load on the
bridge); camera on/off and end-video included.
Direct P2P connects on same / typical networks; a TURN relay for strict-NAT networks is on the roadmap. |
V0+V1 · 15 unit + 11/11 two-user + 10/10 security tests | Built · live |
| Safe to open to beta users | Passwordless email → one-time PIN, signed session, a short-lived WebSocket entry ticket, and per-account daily usage caps. | Resend + Upstash, live | Built · live |
| Stays reachable if a server fails (beta redundancy) | A second, independent bridge runs as a warm standby — same translation models, same shared usage state. Failover is a single endpoint switch (health-aware next). A real cross-host drill confirmed new calls reach the standby in <0.1 s; an in-progress call reconnects rather than migrating. | H0.2 cross-host failover drill, 2026-09-16 | Beta · validated |
| Group calls (3+ people) | Deliberately excluded. Setu is a one-to-one interpreter. Three or more people is a different product (turn-taking, mixing, who-hears-whom) and is out of scope. | scope decision | Out of scope |
| On-device / offline speech-to-text | Speech-to-text is a cloud API today (Gladia, Groq fallback). A local Whisper path exists as a stub for a fully-offline bridge. | — | Roadmap |
| GPU bridge for sub-second turns | Translation and voice run on CPU today. A GPU bridge would push turn latency below a second and raise concurrent-room capacity. | — | Roadmap |
On video connectivity: the peer-to-peer video connects directly on same-network and typical-NAT pairs; a small share of strict-NAT / mobile networks need a TURN relay to connect, which is on the roadmap. Everything above is live on production.
| Item | Note | State |
|---|---|---|
| Real-time two-person translation | Speak/type; six languages, any pair, both ways | Shipped |
| Streaming STT + latency tuning | ~1.5 s/turn; eou 300 ms, beam 2, off-thread | Shipped |
| Reliability & ease (P0/P1) | One status pill, auto-reconnect, control bar, lobby | Shipped |
| Presence, devices, captions (Phases C–D) | Partner card, device picker, notifications, shortcuts | Shipped |
| End-call & accessibility (Phase E) | Leave confirm, end screen, WCAG 2.1 AA | Shipped |
| See each other on video (P2P) | Opt-in camera, invite/accept/decline, live peer-to-peer video alongside translated audio | Shipped |
| Clearer translated conversation | Every turn shows what was said and its translation together; the speaker and their language are always clear; right-to-left scripts are laid out correctly, ready for languages like Urdu or Arabic | Shipped |
| Live speaking feedback | While you talk you see Listening → Speaking → Transcribing in one place, so you always know it heard you | Shipped |
| Easy microphone on phones | Tap to speak — it never listens until you do — with the mic pinned within thumb reach on mobile | Shipped |
| TURN relay for video | Reliable video across strict-NAT / mobile networks | Planned |
| Offline / on-device STT | Local Whisper path for a zero-egress bridge | Planned |
| GPU bridge | Sub-second turns, more concurrent rooms | Planned |
| Group calls (3+) | A different product — turn-taking and mixing | Excluded |
The room UI and access gate are static and cloud-hosted the same way in both lanes. What differs is where the translation bridge runs — flipped with environment variables, never a code change.
Voice detection, translation and speech stay on your own hardware; speech-to-text is a cloud API. Audio is transient — nothing is stored.
The same application on a small cloud box, all models behind API endpoints. Nothing to install; up around the clock.
Today the on-premises bridge lane is what's running (models local, speech-to-text in the cloud, exposed through a tunnel). A fully cloud-hosted twin is a supported shape and is partly there already — speech-to-text is cloud — but is not the deployed lane.
The demo is the actual application, live now. Open it on two devices, sign in with your email and the one-time PIN it sends, and start talking.
