Image Description API Documentation

Table of Contents

1. API Overview

The DescribePicture API allows users to submit an image URL and generate a description based on the provided prompts. This API is especially useful for applications that require image analysis and description generation, such as content moderation, accessibility features, or automated catalog management.

2. API Access Permissions

Before using the DescribePicture API, you need to obtain a valid App ID.

2.1 App ID Application Process

In the API Dashboard, simply provide an email and product name to generate an App ID.

3. Integration Guide

3.1 Request Format

Requests must be sent in JSON format and should include the following fields. Note: If both imageUrl and imageBase64 are provided, the system will prioritize imageUrl over imageBase64.

PromptInfo Object

Each object in the PromptInfo array should contain the following fields:

3.2 Request URL

The base URL for DescribePicture API requests:

https://us-central1-describepicture.cloudfunctions.net/describe_picture_api

3.3 Request Example

Here is a complete Python example using the requests library to send a POST request.

import requests
import json
import base64
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad, unpad
from Crypto.Random import get_random_bytes

# API request URL
url = "https://us-central1-describepicture.cloudfunctions.net/describe_picture_api"

# Key used for request encryption (in hexadecimal format)
hex_key = "690c9d8f65f0d8a8572f1c29e9a30b1d67326281f395b3b649f3bb0afbf19d4b"

# Image URL and MIME type
image_url = "https://blog.hootsuite.com/wp-content/uploads/2023/06/ai-art-prompt-4.png"
mime_type = "image/png"

# Your appId, obtained from the API Dashboard
app_id = "your-app-id"

# Base64-encoded image data (if provided)
image_base64 = ""  # Optional, if Base64 encoded image data is available

# List of prompts to guide the image description content
prompt = [
    {"role": "user", "text": "Describe this picture"},
    {"role": "model", "text": "Describe this picture"},
    {"role": "user", "text": "Describe this picture"}
]

# Function to encrypt the request data
def encrypt_data(data, key):
    # Convert the key from hex to bytes
    key_bytes = bytes.fromhex(key)
    # Generate a random Initialization Vector (IV)
    iv = get_random_bytes(12)
    # Create an AES GCM cipher object
    cipher = AES.new(key_bytes, AES.MODE_GCM, nonce=iv)
    # Encrypt the data using AES GCM mode
    encrypted_data = cipher.encrypt(data.encode())
    # Convert both IV and encrypted data to Base64 and return
    return base64.b64encode(iv).decode(), base64.b64encode(encrypted_data + cipher.digest()).decode()

# Convert request data to JSON format
data = json.dumps({
    "imageUrl": image_url,
    "prompt": prompt,
    "mimeType": mime_type,
    "appId": app_id,
    "imageBase64": image_base64
})

# Call the encryption function to encrypt the data
iv_base64, encrypted_data_base64 = encrypt_data(data, hex_key)

# Prepare the payload for the API request
payload = {
    "iv": iv_base64,              # Initialization Vector (IV) used in encryption
    "encryptedData": encrypted_data_base64  # Encrypted data
}

# Send the POST request to the API
response = requests.post(url, json=payload)

# Print the response status code and content
print("Status Code:", response.status_code)
print("Response:", response.json())

3.4 Response Format

The response is returned in JSON format and includes the following fields:

3.5 Response Example

{
  "answer": "The image shows a sunny beach with three palm trees and a clear blue sky. There is a sign that reads 'Welcome to Paradise'.",
  "lastCredits": 987
}

4. Authentication

To ensure the security of the API, each request must be encrypted using the provided key:

Key: 690c9d8f65f0d8a8572f1c29e9a30b1d67326281f395b3b649f3bb0afbf19d4b

4.1 Data Encryption Example

The following code shows an example of encrypting request data using AES-GCM encryption. This ensures that sensitive information in the request remains secure.

• Encryption Key: We use a hexadecimal encryption key to encode the data.
• Initialization Vector (IV): AES-GCM requires an IV, which we generate as a random 12-byte sequence. The IV is needed to decrypt the data correctly.
• Encryption Process: The data is encrypted, and a unique authentication tag is appended to ensure integrity.

from Crypto.Cipher import AES
from Crypto.Util.Padding import pad, unpad
from Crypto.Random import get_random_bytes
import base64

def encrypt_data(data, key):
    # Convert hexadecimal encryption key to bytes
    key_bytes = bytes.fromhex(key)

    # Generate a 12-byte Initialization Vector (IV) for AES-GCM
    iv = get_random_bytes(12)

    # Create an AES cipher object in GCM mode with the key and IV
    cipher = AES.new(key_bytes, AES.MODE_GCM, nonce=iv)

    # Encrypt the data and calculate the authentication tag for integrity
    encrypted_data = cipher.encrypt(data.encode())

    # Convert both the IV and the encrypted data (with the tag appended) to Base64 for transmission
    iv_base64 = base64.b64encode(iv).decode()
    encrypted_data_base64 = base64.b64encode(encrypted_data + cipher.digest()).decode()

    return iv_base64, encrypted_data_base64

# JSON-formatted data to be encrypted (converted to string format)
data = json.dumps({
    "imageUrl": image_url,
    "prompt": prompt,
    "mimeType": mime_type,
    "appId": app_id,
    "imageBase64": image_base64
})

# Encrypt the data using the provided hex key
iv_base64, encrypted_data_base64 = encrypt_data(data, hex_key)

# The encrypted data and IV are now ready to be sent in the request

5. Error Handling

If a request fails, the API will return a JSON object with an error field describing the nature of the issue. Common errors include:

• Invalid image URL
• Exceeded usage limits
• Unsupported MIME type

6. Quotas and Usage Limits

6.1 Quota Limits

Each appId can send up to 100 requests per hour. If this limit is exceeded, the API will return an error message indicating that the request limit has been reached.

6.2 Purchasing Credits

The DescribePicture API uses a pay-per-use model with a minimum purchase amount of $60. Each request consumes a quota cost of $0.01. Follow these steps to purchase additional credits:

1. Go to the Credit Information section in the API Dashboard to check your current quota.
2. Use the slider to select the desired number of credits; the corresponding cost will be updated automatically.
3. Click Add more credits to be redirected to the payment page.

6.3 Bulk Requests Customization

For large-scale image processing needs, please contact us at aidescribepicture@gmail.com to discuss a customized pricing plan.

6.4 Payment Methods

We accept the following payment methods:

• Credit Cards (Visa, MasterCard, American Express)
• PayPal
• Bank Transfers

7. Customer Support

If you have any questions or need assistance, please contact us at aidescribepicture@gmail.com. We are here to help you make the best use of the DescribePicture API.