Track Cost for Custom Models
Shield360 SDKs ship with a built-in pricing file, so costs are calculated with no network call at start-up. The file, which is regularly updated to reflect the latest pricing from various LLM providers, covers a range of standard LLM usage scenarios and provides detailed pricing information.
To keep SDK costs in step with the prices you manage in Shield360, point pricing_json at your deployment’s public pricing export (https://<your-shield360-host>/api/pricing/export/<db-config-id>), shown as the SDK Pricing URL on Manage Models.
To support personalized pricing schemes for specialized needs, such as fine-tuned or custom models, Shield360 allows users to specify their own pricing structure through a custom JSON file. This flexibility ensures that pricing calculations for LLM operations are accurately aligned with business and application requirements.
Using the Default Pricing File
Section titled “Using the Default Pricing File”The built-in pricing file is recommended for general use as it is consistently updated to stay current with the pricing models of different LLM providers. Users should consider using a custom pricing file primarily when dealing with custom or fine-tuned models not covered by the default settings.
Using a Custom Pricing File
Section titled “Using a Custom Pricing File”Users can provide their own pricing_json variable, which can either be a file path or a URL pointing to a custom JSON file. This file should contain pricing details such as the model names and their associated costs in USD per 1000 tokens for chat-based models.
Example Usage:
-
To load pricing details from an external URL:
shield360.init(pricing_json="https://example.com/path/to/your/pricing.json") -
To load pricing details from a local file path:
shield360.init(pricing_json="/path/to/your/local/pricing.json")
Important Considerations
Section titled “Important Considerations”-
JSON Structure: The custom pricing JSON must follow the same structure as the pricing export from Manage Models. If the structure does not match, Shield360 will not be able to accurately calculate costs based on the custom pricing file.
-
Initialization: The Shield360 SDK loads the provided JSON source at initialization, or uses its built-in file when none is given. It’s essential to ensure that the path or URL is accessible during initialization to fetch the necessary pricing data.
-
Updates: If pricing details change, users need to update their custom JSON file accordingly. If using a URL, ensure that the file at the URL is updated; if using a local file, update the file directly on your system.