Benjamin Paine PRO

Yup! That stays one chunk.

chunker.push("Last week she said, “Hi there. How are you?”");
chunker.flush()

Emitting "Last week she said, “Hi there. How are you?”"

The only exception is with newlines - I wanted it to emit when a newline was encountered.

chunker.push("Last week she said,\n“Hi there. How are you?”");
chunker.flush()

Emitting "Last week she said,"
Emitting "“Hi there. How are you?”"

If you want to disable this behavior, pass in {emitParagraphs: false} to the constructor, i.e.:

const chunker = new SentenceChunker({emitParagraphs: false});

There's also chunkLength to determine the character length maximum (128 by default), and emitTrimmed on whether or not each emit should trim leading/trailing whitespace (default true.) One last thing, if your input is always growing - like if you're streaming one response and just concatenating it as one big string - you can use GrowingSentenceChunker instead (in the same file.) Example:

const chunker = new GrowingSentenceChunker();
chunker.onChunk((chunk) => { console.log(`Emitting "${chunk}"`); });
chunker.push("Last week");
chunker.push("Last week she said");
chunker.push("Last week she said, “Hi there. How are you?”");
chunker.flush()

Emitting "Last week she said, “Hi there. How are you?”"

And just in case it's not obvious, the .flush() call will just emit anything left in the buffer, even if it's shorter than the maximum length. If you don't call .flush(), it will wait for another input that pushes it over the chunk limit before emitting again.

I spent a bit of time working on a JavaScript sentence splitter - it might work right out of the box for this purpose! It tries to split on punctuation when possible for smooth flow, but has a max length option to ensure run-on sentences still get split, too. It also maintains a buffer so you can just keep pushing streaming text into it and it will emit when it has a full chunk.

https://raw.githubusercontent.com/painebenjamin/anachrovox/refs/heads/main/www/sentence.js

Example:

const chunker = new SentenceChunker();
chunker.onChunk((sentenceChunk) => { console.log(`Emitting "${sentenceChunk}"`); });
chunker.push("The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train.");
chunker.flush()

Output:

Emitting "The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration."
Emitting "The best performing models also connect the encoder and decoder through an attention mechanism."
Emitting "We propose a new simple network architecture, the Transformer, based solely on attention mechanisms,"
Emitting "dispensing with recurrence and convolutions entirely."
Emitting "Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train."

Thanks for doing this! I've been all-in on llama.cpp for awhile now but I would be lying if I said I didn't wonder if I was missing out on anything with other engines.

Hello again @JLouisBiz !

I've updated the spaces, they now use Kokoro instead of XTTS. It's drastically faster. Additionally, since the TTS is so much faster, I felt comfortable extended the output to 1024 tokens.

Hello! It's currently clipped at 512 tokens for output, so yes it won't be suitable for very long generation. It's also a very tiny model - Llama 3.2 3B - so definitely more for conversation and less for completing tasks.

I'm going to try and swap in Kokoro TTS which should be faster on these small machines. Thanks for taking the time to test.

I'm sorry that it's not working for you - can you make sure you've given it permission to use your microphone and that you're using the correct one (if you have multiple)? There should be an icon in the corner like this (in chrome) you can click on which should let you select microphones and check levels. Whenever I've had trouble activating it, I've always found I was using the wrong microphone or my voice volume was way far down.

If you're using a browser other than Chrome please let me know, I've tested it in others but there could always be something I'm missing.

Regarding the indicators in the bottom right,

• If the "recording" light doesn't turn on (the top one,) then it did not hear you utter a wake phrase.
• If the "listening" light does turn on, it detects voice activity, but unless you utter a wake phrase it will not send the recording for transcription and completion.

So in short, if you say "Hex Vox, what's the news?" and you don't see the recording light turn on, then it didn't catch the wake phrase and you have to try again.

If instead you just want to speak your command without relying on wake phrase recognition, you can just click the "Call" button - that will start recording immediately and always send the audio for transcription.

This project was the one that set me off on making the wake phrase model in the first place. At first I didn't have it and relied instead on voice activity detection and transcription, however this performs extremely poorly in noisy environments or any kind of muted speech, with near-constant accidental activation. The only efficient way to be always-on AND hands-free was to use a front-end wake-word model to gate the rest of the audio workflow.

You're very welcome! Just so it's clear, the code is licensed under Apache, and the wake-word models are licensed under CC-BY-4.0 (to coincide with the licenses of the audio they were trained on.) More info on the models here: https://huggingface.co/benjamin-paine/anachrovox