Inkling Model Card - Thinking Machines Lab

Description

Inkling is a general-purpose multimodal AI model developed by Thinking Machines Lab, Inc. It features a total of 975 billion parameters, with 41 billion active parameters, and is designed to accept text, image, and audio inputs, generating text outputs. The model is intended for use in English and other languages, making it versatile for various applications. Developers can utilize Inkling for building AI-powered applications, including agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation systems. Its design supports general-purpose conversational use, instruction-following, and other natural language and multimodal tasks. Inkling is released with open weights, allowing for research, fine-tuning, and integration into third-party products by downstream developers.

The architecture of Inkling is based on a 66-layer decoder-only transformer with a sparse Mixture-of-Experts (MoE) feed-forward backbone. Each token is routed to six of 256 experts, with two shared experts active on every token. The model is natively multimodal, encoding images via a hierarchical patch encoder and audio through discrete token encoding. All modalities are projected into a shared hidden space and processed jointly by the decoder. Inkling supports a context window of up to 1 million tokens, making it capable of handling extensive input data.

Inkling is available through API access via Tinker, the service for fine-tuning, and through third-party inference providers. The model's weights can be downloaded from Hugging Face. For optimal performance, specific hardware configurations are required, including GPU clusters with substantial VRAM. The model has undergone extensive safety evaluations to mitigate risks associated with multimodal interactions, ensuring that it does not present material uplift beyond existing open-weight models. However, users are advised to implement additional safeguards and human oversight when deploying Inkling in high-stakes contexts.

Inkling Model Card Highlights

• Model Type: Multimodal autoregressive transformer

• API Available: Yes

• License: Apache 2.0

• Parameters: 975B total, 41B active

• Fine-tuning Support: Yes

• Multimodal: Yes

• Numerics Support: BF16, MXFP8, NVFP4

• Context Window: 1M tokens

Getting Started with Inkling Model Card

1. 1

   Access model: Visit the Tinker service for API access.

2. 2

   Authenticate: Set up your account and obtain API keys.

3. 3

   Set up environment: Ensure your hardware meets the model's requirements.

4. 4

   Integrate via API: Use the provided documentation to integrate Inkling into your application.

5. 5

   Optimize: Fine-tune the model as needed for your specific use case.

Inkling Model Card's Use Cases

• Chatbots
• Coding Assistants
• Multimodal Applications
• Instruction-Following
• Retrieval-Augmented Generation

FAQ from Inkling Model Card

• What is Inkling?

  Inkling is a multimodal AI model that accepts text, image, and audio inputs to generate text outputs.

• What are the intended uses of Inkling?

  Inkling is intended for use in applications such as chatbots, coding assistants, and agentic systems.

• What is the architecture of Inkling?

  Inkling is based on a 66-layer decoder-only transformer with a sparse Mixture-of-Experts backbone.

• How many parameters does Inkling have?

  Inkling has a total of 975 billion parameters, with 41 billion active.

• Is Inkling available for fine-tuning?

  Yes, Inkling supports fine-tuning and is released with open weights.

• What hardware is required to run Inkling?

  Inkling requires a GPU cluster with at least 2 TB of aggregated VRAM for the BF16 checkpoint.

• Where can I access Inkling's weights?

  The weights for Inkling can be downloaded from Hugging Face.

• What safety measures are in place for Inkling?

  Inkling has undergone safety evaluations to mitigate risks associated with multimodal interactions.