Phi-4 & Phi-4 Multimodal

Deploy Microsoft Phi-4 and Phi-4-multimodal on Spheron GPU instances using vLLM. Phi-4 is a 14B parameter small language model (SLM) with strong reasoning capabilities released under the MIT license.

Recommended hardware

Model	Recommended GPU	Instance Type	Notes
Phi-4 (14B)	A100 40GB	Dedicated	Full precision
Phi-4-multimodal	RTX 4090 (24GB) or A100	Dedicated	Image + text input

Manual setup

Use these steps to set up the server manually after SSH-ing into your instance. This works on any provider regardless of cloud-init support.

Step 1: Connect to your instance

ssh <user>@<ipAddress>

Replace <user> with the username shown in the instance details panel (e.g., ubuntu for Spheron AI instances) and <ipAddress> with your instance's public IP.

Step 2: Install vLLM

sudo apt-get update -y
sudo apt-get install -y python3-pip
pip install vllm

Step 3: Start the server

Run the server in the foreground to verify it works:

python3 -m vllm.entrypoints.openai.api_server \
  --model microsoft/phi-4 \
  --port 8000 \
  --dtype bfloat16 \
  --trust-remote-code

Press Ctrl+C to stop. For Phi-4-multimodal, replace microsoft/phi-4 with microsoft/Phi-4-multimodal-instruct.

Step 4: Run as a background service

To keep the server running after you close your SSH session, create a systemd service:

sudo tee /etc/systemd/system/vllm-phi4.service > /dev/null << 'EOF'
[Unit]
Description=Phi-4 vLLM Inference Server
After=network.target
 
[Service]
Type=simple
ExecStart=/usr/bin/python3 -m vllm.entrypoints.openai.api_server \
  --model microsoft/phi-4 \
  --port 8000 \
  --dtype bfloat16 \
  --trust-remote-code
Restart=on-failure
RestartSec=10
 
[Install]
WantedBy=multi-user.target
EOF
 
sudo systemctl daemon-reload
sudo systemctl enable vllm-phi4
sudo systemctl start vllm-phi4

Accessing the server

SSH tunnel

ssh -L 8000:localhost:8000 <user>@<ipAddress>

Usage example: text

from openai import OpenAI
 
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
 
response = client.chat.completions.create(
    model="microsoft/phi-4",
    messages=[{"role": "user", "content": "Write a Python function that checks if a number is prime."}],
)
print(response.choices[0].message.content)

Usage example: image input (Phi-4-multimodal)

import base64
from openai import OpenAI
 
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
 
with open("image.jpg", "rb") as f:
    image_b64 = base64.b64encode(f.read()).decode()
 
response = client.chat.completions.create(
    model="microsoft/Phi-4-multimodal-instruct",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_b64}"}},
                {"type": "text", "text": "What is shown in this image?"},
            ],
        }
    ],
)
print(response.choices[0].message.content)

Cloud-init startup script (optional)

If your provider supports cloud-init, you can paste this into the Startup Script field when deploying to automate the setup above.

Phi-4 (A100 40GB)

#cloud-config
runcmd:
  - apt-get update -y
  - apt-get install -y python3-pip
  - pip install vllm
  - |
    cat > /etc/systemd/system/vllm-phi4.service << 'EOF'
    [Unit]
    Description=Phi-4 vLLM Inference Server
    After=network.target
 
    [Service]
    Type=simple
    ExecStart=/usr/bin/python3 -m vllm.entrypoints.openai.api_server \
      --model microsoft/phi-4 \
      --port 8000 \
      --dtype bfloat16 \
      --trust-remote-code
    Restart=on-failure
    RestartSec=10
 
    [Install]
    WantedBy=multi-user.target
    EOF
  - systemctl daemon-reload
  - systemctl enable vllm-phi4
  - systemctl start vllm-phi4

Phi-4-multimodal

Replace the model in ExecStart with microsoft/Phi-4-multimodal-instruct.

What's next

Gemma 3: Another efficient small model option (Gemma Terms of Use)
Multimodal Models: Other vision-language model guides
vLLM Inference Server: vLLM configuration details
Instance Types: GPU selection for SLMs