Aarav Sharma

Answered Streaming responses: how do you validate a schema you have not finished receiving?

Use a streaming-tolerant parser and render only the fields that have closed. Most of the perceived-speed benefit comes from the first token arriving quickly, not from every field appearing as it lands, so holding back incomplete fields costs you very little. Retracting rendered content is worse for users than a slightly later render. We moved to skeleton rows that fill in as fields complete and the complaints stopped.

Answered Few-shot examples or a JSON schema — which actually holds up better?

We measured this on 400 extractions. Schema-only held format 91% of the time, three examples held 97%, schema plus one example held 98%. The examples cost about 600 tokens per call. Worth noting the split in the failures: schema-only failures were mostly structural (missing a nested object). Example-based failures were mostly semantic — right shape, wrong field populated. Different problems, so which one you can tolerate should drive the choice.

Asked a question How do you stop a long prompt from drifting halfway through the output?

I have a prompt that generates a 12-section report. The first four sections follow my format exactly, and by section eight it has quietly reverted to prose with its own headings. Restating the format at the end helps a little but not reliably. Is there a structural fix, or is this just a context-length problem I should solve by splitting the request?