A Unified Jev-Style Wrapper for LLMs and Vision Models

Posted by Allan Riordan Boll on Friday, September 25, 2026

I've recently been fascinated by Jev and the ecosystem of self-hosted alternatives emerging around it, such as SemIf and OpenJev. While exploring these, I discovered a particularly elegant technique: extracting an LLM's token probabilities.

The Core Mechanism: Logprobs

The fundamental strategy involves structuring a prompt to force a multiple-choice response. For example:

State: My order arrived broken and I want a refund.

[A] billing [B] shipping [C] returns

Answer with the letter of the best option only.

To implement this, you pass specific parameters in a Chat Completions request: { "max_completion_tokens": 1, "logprobs": true, "top_logprobs": 20 }

By restricting the model to a single token, you eliminate verbose rambling and significantly increase speed, although the initial input processing still incurs a time cost. If the backend supports it, a shared state prefix can be $\text{KV-cached}$ to further optimize performance.

Extending Jev for Vision

While the standard Jev request format is limited to text and JSON, I decided to expand it for my own tests by adding an attachments field for images.

I built a prototype that captures webcam frames as base64 JPEGs and evaluates them based on a set of criteria. The system answers three specific questions per frame:

1. Is a human present?
2. Is the environment internal or external?
3. What is the current lighting level?

Performance Benchmarks

The efficiency varies wildly depending on the hardware and API:

The slower speed of the OpenAI model is likely due to the overhead of establishing a separate connection for every single question per frame. While specialized CV models are undoubtedly more efficient, the primary advantage here is flexibility: you can modify a detection condition simply by rewriting a sentence in plain English.

System Workflow

Implementation Details

To get this running, you'll need the following:

• Python environment
• uv for dependency management
• opencv-python (for webcam access)
• Access to a compatible LLM API (llama.cpp or OpenAI)

Below is the complete standalone Python implementation:

#!/usr/bin/env -S uv run --script 
# /// script 
# dependencies = [ opencv-python ] 
# /// 
# Preview and score webcam frames with llama.cpp or OpenAI.

import argparse 
import base64 
import concurrent.futures 
import datetime 
import json 
import math 
import mimetypes 
import os 
import pathlib 
import time 
import urllib.parse 
import urllib.request 
import cv2 

# Custom extension to Jev request format including 'attachments'
request_data = {
    "state": "Webcam Feed",
    "attachments": [], 
    "questions": { 
        "person": { "type": "noul", "instructions": "Is a person visible?" }, 
        "plant": { "type": "noul", "instructions": "Is a plant visible?" }, 
        "setting": { 
            "type": "choice", 
            "instructions": "Where is the camera?", 
            "criteria": { "indoors": None, "outdoors": None, "unclear": None } 
        }, 
        "light": { 
            "type": "score", 
            "instructions": "How bright is the scene?", 
            "criteria": ["dark", "dim", "bright"] 
        } 
    } 
}

def score(data, url, model): 
    state = data["state"] 
    if not isinstance(state, str): 
        state = json.dumps(state) 

    images = [] 
    for attachment in data.get("attachments", []): 
        if attachment.startswith("data:image/"): 
            images.append(attachment) 
            continue 
        path = pathlib.Path(attachment).expanduser() 
        mime_type, _ = mimetypes.guess_type(path) 
        if mime_type not in {"image/png", "image/jpeg", "image/webp", "image/gif"}: 
            raise ValueError(f"Unsupported image file: {path}") 
        encoded = base64.b64encode(path.read_bytes()).decode() 
        images.append(f"data:{mime_type};base64,{encoded}") 

    is_openai = urllib.parse.urlsplit(url).hostname == "api.openai.com" 
    headers = { "Content-Type": "application/json" } 
    if is_openai: 
        headers["Authorization"] = "Bearer " + os.environ["OPENAI_API_KEY"] 

    answers = {} 
    for name, question in data["questions"].items(): 
        if question["type"] == "choice": 
            options = question["criteria"] 
        elif question["type"] == "noul": 
            options = {"true": None, "false": None} | question.get("criteria", {}) 
        elif question["type"] == "score": 
            options = {str(i): description for i, description in enumerate(question["criteria"])} 
        else: 
            raise ValueError(f"Unknown question type: {question['type']}") 

        if not 2 <= len(options) <= 20: 
            raise ValueError("Provide 2 to 20 criteria per question.") 

        letters = "ABCDEFGHIJKLMNOPQRST"[:len(options)] 
        instructions = question["instructions"] 
        if not isinstance(instructions, str): 
            instructions = json.dumps(instructions) 

        lines = [f"State:\n{state}\n\nQuestion: {instructions}\nOptions: "] 
        for letter, (key, description) in zip(letters, options.items()): 
            line = f"[{letter}] {key}" 
            if description is not None: 
                line += f": {description}" 
            lines.append(line) 
        
        prompt = "\n".join(lines) + "\n\nAnswer with the letter of the best option only." 

        if is_openai: 
            endpoint = "/responses" 
            content = [{"type": "input_text", "text": prompt}] 
            content.extend({"type": "input_image", "image_url": image} for image in images) 
            body = { 
                "model": model, 
                "input": [{"role": "user", "content": content}], 
                "reasoning": {"effort": "none"}, 
                "max_output_tokens": 16, 
                "top_p": 1, 
                "top_logprobs": 20, 
                "include": ["message.output_text.logprobs"], 
            } 
        else: 
            endpoint = "/chat/completions" 
            content = [{"type": "text", "text": prompt}] 
            content.extend({"type": "image_url", "image_url": {"url": image}} for image in images) 
            body = { 
                "model": model, 
                "messages": [{"role": "user", "content": content}], 
                "max_completion_tokens": 1, 
                "temperature": 0, 
                "reasoning_effort": "none", 
                "logprobs": True, 
                "top_logprobs": 1024, 
            } 

        request = urllib.request.Request( 
            url.rstrip("/") + endpoint, 
            headers=headers, 
            data=json.dumps(body).encode(), 
        ) 
        with urllib.request.urlopen(request) as response: 
            result = json.load(response) 
            # Logic to parse result logprobs would follow here...

Visual Examples

Below is a sample of the webcam input being processed by the wrapper: