Synthetic data
Generate, transform, validate, and refine datasets through explicit stages with separate quality checks.
Ploinky Worker turns repeatable LLM jobs into small, deliberate pipelines. Match every step with the right-sized model, run against local or remote infrastructure, and keep throughput, cost, and results visible.
# queue a repeatable LLM task $ pworker run ./profile.mjs \ --input '{"page":"report.html"}' \ --async { "status": "queued", "phase": "extract" } # inspect it without holding the process $ pworker --status 9f3… { "status": "running", "phase": "classify" }
Use Ploinky Worker when the shape of the task is known but the inputs keep changing. Each run follows an inspectable flow instead of starting an open-ended agent loop.
Generate, transform, validate, and refine datasets through explicit stages with separate quality checks.
Run LLM-backed test suites and evals as background jobs, then inspect the current phase and persisted result.
Extract signals, segment audiences, compare messages, and produce structured profiles across many inputs.
Break long documents or pages into controlled extraction, classification, synthesis, and validation steps.
Queue many independent jobs, group compatible requests, and route each answer back to the correct task.
Mix model calls with JavaScript steps that read, transform, and write files in a caller-selected directory.
A task is a readable sequence of phases. Each phase can request a logical capability tier, perform a deterministic JavaScript operation, or finish with a structured result. The actual model behind a tier is configured once and can point to a remote provider or local endpoint.
This lets routine extraction stay on a small model while nuanced synthesis or validation moves to a stronger one—without hard-coding a provider into every task.
Ploinky Worker is designed to keep useful work flowing through the model infrastructure you already have—from a developer machine to cloud or shared GPU-backed endpoints.
Pack eligible phase requests together while keeping every task ID, result, and continuation independent.
Route by tier, throttle requests, cache complete repeats, and observe request counts and transferred bytes.
Queue repeatable jobs so capable local or remote schedulers have a steady stream of appropriately sized work.
Clear boundary: prompt batching can reduce repetitive requests, but token savings, latency gains, and GPU saturation depend on the selected model server, provider, scheduler, and workload. Ploinky Worker supplies the task, queue, tier, and dispatch structure needed to optimize them deliberately.
Submit work synchronously or return a task ID immediately. Check the active phase later, retrieve the final result, and tune the pipeline using observable runtime behavior.
Queued work can continue independently while clients poll for status, completion, or a bounded error.
Connect provider APIs, OpenAI-compatible endpoints, or configured local models behind the same task format.
Readable task files and explicit phase transitions make workflows easier to review, test, and improve.
Ploinky Worker is open source. Explore the task format, CLI, provider routing, local models, batching, monitoring, and background execution today. For advanced deployments, the team offers consulting on pipeline design, provider topology, batching strategy, shared GPU infrastructure, and local inference integration.