Deep Dive into Nvidia Nemotron 3.5 Lightning

The Nvidia Nemotron 3.5 Lightning is a high-performance model hosted on Hugging Face under the identifier nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4. This model is designed for efficiency and precision, utilizing a specific quantization and architecture to balance power and speed.

🛠 Technical Specifications

The model employs a specific set of special tokens to manage conversation boundaries and unknown inputs. These are critical for the tokenizer to correctly parse the input stream.

The model's scale is approximately $30\text{B}$ parameters, optimized via the NVFP4 format for accelerated inference.

---

💬 The Chat Template Logic

The core of the model's interaction is governed by a complex Jinja2 chat template. This template ensures that the model receives prompts in a structured format that it can interpret reliably.

Message Flow Architecture

The following diagram illustrates how the template processes different message roles:

Key Template Features

• render_extra_keys Macro: This internal function handles the conversion of JSON dictionaries into a readable string format, ensuring that any metadata not explicitly handled by the main logic is still preserved in the prompt.
• Thinking Process: The model supports a "Chain of Thought" mechanism. If enable_thinking is active, the assistant's internal reasoning is wrapped in <think> and </think> tags.
• History Management: The template tracks the last_user_idx to determine whether to truncate previous reasoning content to save context window space.

---

🔧 Tool Use and Function Calling

One of the most powerful aspects of Nemotron 3.5 Lightning is its structured approach to tool integration. When tools are provided, the system prompt is automatically augmented with a detailed list of available functions.

Function Definition Format

Each tool is described using a pseudo-XML structure:

• <function>: The wrapper for the tool.
• <name>: The unique identifier of the function.
• <description>: A clear explanation of what the tool does.
• <parameters>: A detailed list of required and optional inputs, including their type and enum values.

The Execution Protocol

If the model decides to invoke a tool, it must adhere to a strict formatting requirement. It cannot use standard JSON; instead, it uses a custom tag-based system:

Required Tool Call Format:

tool_call>
function=example_function_name>
parameter=example_parameter_1>
value_1
/parameter>
parameter=example_parameter_2>
Value for the second parameter
/parameter>
/function>
/tool_call>

Critical Constraints for Tool Use:

• Function calls must be nested within tool_call> tags.
• All required parameters must be present.
• Reasoning may be provided before the call, but never after.
• If no tool is applicable, the model should respond normally without mentioning the tool-calling mechanism.

---

🖼 Resources and Metadata

Below are the official links and identifiers associated with the model's ecosystem:


License Note: This model is released under the OpenMDW-1.1 license.