# Shishir Poudel — full profile > ML engineer in Nepal. I build real-time voice AI (speech recognition → language model → speech synthesis) that serves 1,000+ concurrent conversations on self-hosted infrastructure in Nepal, retrieval with locally deployed LLMs, and the product systems around them. - Name: Shishir Poudel (handle: Shishir3D) - Role: ML engineer, based in Nepal - Email: shishirpoudel.dev@gmail.com - GitHub: https://github.com/Shishir3D - LinkedIn: https://www.linkedin.com/in/shishir3d/ - Résumé: https://www.shishir-poudel.com.np/resume/ - Website: https://www.shishir-poudel.com.np/ ## Experience ### AI Developer, Next AI (Nov 2025 — now) Real-time voice AI and speech models. Fine-tuned Gemma 3 and Piper; deployed OmniVoice and Qwen3-TTS; building local Nepali STT, LLM and TTS. ### Full-Stack Developer (contract), Simal (Jun 2025 — now) ERP, CRM and freight dispatch apps in React Native, Supabase and PostgreSQL. ## Projects ### 01. Real-time voice AI (Next AI) Speak to it, it talks back. A streaming speech pipeline I built that serves 1,000+ concurrent conversations on self-hosted infrastructure in Nepal. Key facts: - < 700 ms: voice-to-voice target - 1,000+: concurrent sessions - Self-hosted: own infrastructure, in Nepal Latency budget per turn (Target budget, not a benchmark. Every stage streams, so the next starts before the last finishes.): - In: 40 ms - STT: 150 ms - LLM 1st token: 300 ms - TTS 1st audio: 120 ms - Out: 40 ms What I did: - Designed and built the STT → LLM → TTS pipeline and its orchestration - Fine-tuned Piper (TTS) and Gemma 3; deployed OmniVoice and Qwen3-TTS - Run it self-hosted in Nepal for 1,000+ concurrent users Architecture, in the order data moves: - User's phone (mobile app · mic + speaker): The app streams microphone audio and plays the reply. Part of an app with 500,000+ downloads; I built the voice system, not the whole app. - LiveKit room (WebRTC): Each conversation is a real-time room. WebRTC keeps audio low-latency in both directions. - Speech → text (Whisper · Nemotron · IndicConformer): Streaming transcription. Partial transcripts flow on before the user finishes speaking. IndicConformer covers Nepali and other Indic speech. - Language model (local Gemma · Gemini · DeepSeek): Generates the reply token by token. A locally served Gemma keeps traffic on our own machine; hosted models are used where they fit better. - Text → speech (OmniVoice · Qwen3-TTS · Kokoro · Piper): Turns the first sentence into audio as soon as it exists. Piper is fine-tuned by me. Built with: Python, FastAPI, LiveKit, WebRTC, Whisper, Gemma, Qwen3-TTS, Piper. See it: https://www.shishir-poudel.com.np/2/ (3D model), https://www.shishir-poudel.com.np/3/#voice (film), https://www.shishir-poudel.com.np/4/#voice (temple garden). ### 02. Customer-support AI (Next AI) Answers support questions from the company's own documents, using a language model that runs on our own hardware. Key facts: - Local: self-hosted LLM - Grounded: answers from retrieved docs - Gemma 3: fine-tuned for support What I did: - Built the retrieval workflow over the support knowledge base - Deployed local LLM inference - Fine-tuned Gemma 3 for support replies Architecture, in the order data moves: - Customer question (chat): A customer asks something in their own words. - Support docs (chunked + embedded): Help articles and policies, split into chunks and embedded once, ahead of time. - Retriever (vector search): Finds the few chunks most relevant to the question. - Prompt builder (question + top chunks): Puts the question and the retrieved chunks into one prompt, so the model answers from sources rather than memory. - Local LLM (fine-tuned Gemma 3): Runs on our own machine. Fine-tuned for support tone and format. - Answer (back to chat): The reply goes back to the customer. Built with: Python, RAG, Embeddings, Vector search, Gemma 3. See it: https://www.shishir-poudel.com.np/2/ (3D model), https://www.shishir-poudel.com.np/3/#rag (film), https://www.shishir-poudel.com.np/4/#rag (temple garden). ### 03. Manufacturing ERP (Product work) Factory staff log production on a phone, even with no internet. Data syncs when they're back online, separated per company. Key facts: - Offline: works with no signal - Paged sync: push / pull - Per-tenant: row-level security What I did: - Built production workflows and BOM-driven dynamic forms - Worked on offline storage and paged push / pull sync - Built configurable reports and a generative dashboard Architecture, in the order data moves: - Form blueprint (JSON + bill of materials): Forms are described in JSON. Picking a product's bill of materials pre-fills the inputs. - Mobile app (React Native · Expo): Renders the form and validates entries on the factory floor. - On-device DB (WatermelonDB): Every entry is saved locally first, so nothing is lost without a connection. - Sync (push / pull in pages): When online, local changes are pushed and remote changes pulled in pages, dependent records in order. - PostgreSQL (Supabase · RLS per tenant): Row-level security keeps each company's data separate at the database. - Reports (validated dashboard specs): Reports and generated dashboards, checked against a component catalog before rendering. Built with: React Native, Expo, TypeScript, WatermelonDB, Supabase, PostgreSQL. See it: https://www.shishir-poudel.com.np/2/ (3D model), https://www.shishir-poudel.com.np/3/#erp (film), https://www.shishir-poudel.com.np/4/#erp (temple garden). ### 04. Multi-company CRM (Product work) One CRM app serving many businesses. Each company only ever sees its own people, deals and tasks. Key facts: - Multi-org: one app, many companies - Roles: per workspace - Web + mobile: one codebase What I did: - Built React Native / Expo workflows and shared UI - Built org-scoped query state and typed feature APIs - Worked on auth, workspace roles and row-level security Architecture, in the order data moves: - Web & mobile app (Expo Router): People, companies, deals and tasks on phone and browser. - Client cache (TanStack Query · keyed by org): Cached data is keyed by the active company, so switching companies never shows the wrong data. - Auth & roles (workspace roles): Who you are and what your role allows in this company. - PostgreSQL (row-level security): The database enforces access itself, independent of the app. - Company A data (isolated rows): Only members of company A can read these rows. - Company B data (isolated rows): Only members of company B can read these rows. Built with: React Native, Expo Router, TanStack Query, Supabase, PostgreSQL, Zod. See it: https://www.shishir-poudel.com.np/2/ (3D model), https://www.shishir-poudel.com.np/3/#crm (film), https://www.shishir-poudel.com.np/4/#crm (temple garden). ### 05. Freight dispatch (In development) Factories request a truck, transport operators assign one, drivers check in at each stop. Everyone sees the same trip. Key facts: - 3 roles: factory · operator · driver - 4 checkpoints: per delivery - Typed: validated commands What I did: - Built load requests and truck / driver assignment - Connected typed RPC commands and validated forms to Supabase - Built shared trip state with revision checks Architecture, in the order data moves: - Factory (load request): Enters cargo, pickup and delivery details. - Transport operator (assign truck + driver): Picks a truck and driver for the load. - Commands (typed RPC · revision check): Every change is a validated command. A revision check stops two people overwriting each other; retry IDs stop duplicates. - Shared trip (PostgreSQL · per-party access): One trip record; each party sees only what its role allows. - Driver app (checkpoints): Heading to pickup → at pickup → loaded → delivered. Checkpoints, not live GPS. Built with: React Native, Expo, TypeScript, Supabase RPC, PostgreSQL, Zod. See it: https://www.shishir-poudel.com.np/2/ (3D model), https://www.shishir-poudel.com.np/3/#dispatch (film), https://www.shishir-poudel.com.np/4/#dispatch (temple garden). ## Skills - Speech: STT, TTS, fine-tuning, streaming pipelines - LLMs: RAG, local inference, Gemma, structured output - Real-time: LiveKit, WebRTC, WebSockets, FastAPI - Product: React Native, TypeScript, PostgreSQL - Infra: Linux VMs, Docker, CI/CD ## Recognition - NASA Space Apps 2023 — Honourable Mention - Hackathon winner - AWS Certified Cloud Practitioner ## Side builds - Plant Monitoring System — ESP32 sensors with automatic watering. (https://github.com/Shishir3D/PlantMonitoringSystem) - PyQuest — Learning game with Gemini-generated questions. (https://github.com/Shishir3D/pyquest) - CPR Feedback — Arduino device that graphs compression depth and rate. (https://github.com/Shishir3D/CPR-Feedback) ## Versions of the portfolio - Voice home: https://www.shishir-poudel.com.np/ — Real-time voice AI that listens and answers out loud, with swipeable demos of each project. - Illustrated portfolio: https://www.shishir-poudel.com.np/1/ — The story told as an illustrated, animated journey: a frog climbing out of a well. - Systems in 3D: https://www.shishir-poudel.com.np/2/ — Each system's real architecture as a 3D model you can step through, one component at a time. - Systems in motion: https://www.shishir-poudel.com.np/3/ — Each system as a short, scrubbable film of one request moving through it. - Temple garden: https://www.shishir-poudel.com.np/4/ — The work as a 3D Japanese temple garden at dusk; each building holds one system.