LangWatch

LangWatch

LangWatch is an open-source LLMOps platform designed to monitor, evaluate, debug, and optimize large language model (LLM) applications and AI agents. It provides observability into AI workflows, capturing detailed traces, conversations, token usage, and costs, enabling teams to collaborate and improve AI performance efficiently.

Use tool

Pricing

Developer
$/mo
  • LLM Monitoring & Tracing: Traces & Graphs (Agents)
  • LLM Monitoring & Tracing: Threads Tracking (Conversations/Sessions)
  • LLM Monitoring & Tracing: User Tracking
  • LLM Monitoring & Tracing: Cost and Token Tracking
  • LLM Monitoring & Tracing: Triggers & Alerts (Slack, Email)
  • LLM Monitoring & Tracing: SDKs (Python, Typescript)
  • LLM Monitoring & Tracing: OpenTelemetry (Java, Go, custom)
  • LLM Monitoring & Tracing: Custom REST API integration
  • LLM Monitoring & Tracing: Included Usage: 1k traces
  • LLM Monitoring & Tracing: Retention: 30 days
  • Evaluations & Prompt Optimization: Real-time Evaluations & Guardrails
  • Evaluations & Prompt Optimization: Offline Evaluation (CI/CD, Notebooks and Workflows Experimentation)
  • Evaluations & Prompt Optimization: DSPy Prompt Optimization
  • Evaluations & Prompt Optimization: Evaluation Wizard
  • Evaluations & Prompt Optimization: Access to 30+ Evaluators in our Library
  • Data Flywheel: Datasets
  • LangWatch Analytics: User-Analytics, Topic Detection, Sentiment Analysis, Feedback
  • LangWatch Analytics: Customizable graphs on any metric available in the platform
  • Collaboration: Projects: Unlimited
  • Collaboration: Users: 2
  • API: Extensive Public API
  • Support: Community (GitHub, Discord)
  • Billing & Compliance: Subscription Management: Self-serve
  • Billing & Compliance: Payment Methods: Credit card
  • Billing & Compliance: Contract Duration: Monthly
  • Billing & Compliance: Contracts: Standard T&Cs
Launch
$/mo
  • LLM Monitoring & Tracing: Traces & Graphs (Agents)
  • LLM Monitoring & Tracing: Threads Tracking (Conversations/Sessions)
  • LLM Monitoring & Tracing: User Tracking
  • LLM Monitoring & Tracing: Cost and Token Tracking
  • LLM Monitoring & Tracing: Triggers & Alerts (Slack, Email)
  • LLM Monitoring & Tracing: SDKs (Python, Typescript)
  • LLM Monitoring & Tracing: OpenTelemetry (Java, Go, custom)
  • LLM Monitoring & Tracing: Custom REST API integration
  • LLM Monitoring & Tracing: Included Usage: 20k traces
  • LLM Monitoring & Tracing: Retention: 180 days
  • Evaluations & Prompt Optimization: Real-time Evaluations & Guardrails
  • Evaluations & Prompt Optimization: Offline Evaluation (CI/CD, Notebooks and Workflows Experimentation)
  • Evaluations & Prompt Optimization: DSPy Prompt Optimization
  • Evaluations & Prompt Optimization: Evaluation Wizard
  • Evaluations & Prompt Optimization: Access to 30+ Evaluators in our Library
  • Evaluations & Prompt Optimization: Custom Experiments (via SDK)
  • Evaluations & Prompt Optimization: External Evaluation Pipelines
  • Data Flywheel: Datasets
  • Data Flywheel: Auto-Building Datasets (from real-time trace filters and automated LLM evaluations)
  • Data Flywheel: User Feedback Collection
  • Data Flywheel: Human Annotation
  • LangWatch Analytics: User-Analytics, Topic Detection, Sentiment Analysis, Feedback
  • LangWatch Analytics: Customizable graphs on any metric available in the platform
  • LangWatch Analytics: Tracking functional KPIs, allowing stakeholders to visualize performance metrics in real time
  • LangWatch Analytics: Trend analysis and performance benchmarking
  • LangWatch Analytics: Detailed tracking of costs including per-request costs and overall operational expenses
  • Collaboration: Projects: Unlimited
  • Collaboration: Users: 3
  • API: Extensive Public API
  • Support: Community (GitHub, Discord)
  • Support: Chat & Email
  • SSO via Google, AzureAD, GitHub (Microsoft)
  • RBAC
  • Billing & Compliance: Subscription Management: Self-serve
  • Billing & Compliance: Payment Methods: Credit card
  • Billing & Compliance: Contract Duration: Monthly
  • Billing & Compliance: Contracts: Standard T&Cs
Accelerate
$/mo
  • LLM Monitoring & Tracing: Traces & Graphs (Agents)
  • LLM Monitoring & Tracing: Threads Tracking (Conversations/Sessions)
  • LLM Monitoring & Tracing: User Tracking
  • LLM Monitoring & Tracing: Cost and Token Tracking
  • LLM Monitoring & Tracing: Triggers & Alerts (Slack, Email)
  • LLM Monitoring & Tracing: SDKs (Python, Typescript)
  • LLM Monitoring & Tracing: OpenTelemetry (Java, Go, custom)
  • LLM Monitoring & Tracing: Custom REST API integration
  • LLM Monitoring & Tracing: Included Usage: 20k traces
  • LLM Monitoring & Tracing: Retention: Up to 1 year
  • Evaluations & Prompt Optimization: Real-time Evaluations & Guardrails
  • Evaluations & Prompt Optimization: Offline Evaluation (CI/CD, Notebooks and Workflows Experimentation)
  • Evaluations & Prompt Optimization: DSPy Prompt Optimization
  • Evaluations & Prompt Optimization: Evaluation Wizard
  • Evaluations & Prompt Optimization: Access to 30+ Evaluators in our Library
  • Evaluations & Prompt Optimization: Custom Experiments (via SDK)
  • Evaluations & Prompt Optimization: External Evaluation Pipelines
  • Evaluations & Prompt Optimization: LLM-as-judge Evaluators
  • Evaluations & Prompt Optimization: Build your own Eval
  • Data Flywheel: Datasets
  • Data Flywheel: Auto-Building Datasets (from real-time trace filters and automated LLM evaluations)
  • Data Flywheel: User Feedback Collection
  • Data Flywheel: Human Annotation
  • Data Flywheel: Human Annotation Queues
  • Data Flywheel: Automated LLM-as-a-judge Augmenting
  • Data Flywheel: Export all Traces and Data
  • LangWatch Safeguards: Jailbreaking / Prompt Injection
  • LangWatch Safeguards: Business Sensitive evaluation
  • LangWatch Safeguards: PII detection and auto-redaction
  • LangWatch Safeguards: Competitor blocklist, off-topic evaluation
  • LangWatch Safeguards: Content Moderation
  • LangWatch Safeguards: Custom Guardrails
  • LangWatch Analytics: User-Analytics, Topic Detection, Sentiment Analysis, Feedback
  • LangWatch Analytics: Customizable graphs on any metric available in the platform
  • LangWatch Analytics: Tracking functional KPIs, allowing stakeholders to visualize performance metrics in real time
  • LangWatch Analytics: Trend analysis and performance benchmarking
  • LangWatch Analytics: Detailed tracking of costs including per-request costs and overall operational expenses
  • Collaboration: Projects: Unlimited
  • Collaboration: Users: 5
  • API: Extensive Public API
  • Support: Community (GitHub, Discord)
  • Support: Chat & Email
  • Support: Private Slack/Discord Channel
  • SSO via Google, AzureAD, GitHub (Microsoft)
  • RBAC
  • Enterprise SSO (Okta, AzureAD/EntraID)
  • Billing & Compliance: Subscription Management: Account Manager
  • Billing & Compliance: Payment Methods: Credit card, Invoice
  • Billing & Compliance: Contract Duration: Custom
  • Billing & Compliance: Contracts: Standard T&Cs
  • Billing & Compliance: Data Processing Agreement (GDPR)
Enterprise
$/mo
  • LLM Monitoring & Tracing: Traces & Graphs (Agents)
  • LLM Monitoring & Tracing: Threads Tracking (Conversations/Sessions)
  • LLM Monitoring & Tracing: User Tracking
  • LLM Monitoring & Tracing: Cost and Token Tracking
  • LLM Monitoring & Tracing: Triggers & Alerts (Slack, Email)
  • LLM Monitoring & Tracing: SDKs (Python, Typescript)
  • LLM Monitoring & Tracing: OpenTelemetry (Java, Go, custom)
  • LLM Monitoring & Tracing: Custom REST API integration
  • LLM Monitoring & Tracing: Included Usage: Custom
  • LLM Monitoring & Tracing: Retention: Custom
  • Evaluations & Prompt Optimization: Real-time Evaluations & Guardrails
  • Evaluations & Prompt Optimization: Offline Evaluation (CI/CD, Notebooks and Workflows Experimentation)
  • Evaluations & Prompt Optimization: DSPy Prompt Optimization
  • Evaluations & Prompt Optimization: Evaluation Wizard
  • Evaluations & Prompt Optimization: Access to 30+ Evaluators in our Library
  • Evaluations & Prompt Optimization: Custom Experiments (via SDK)
  • Evaluations & Prompt Optimization: External Evaluation Pipelines
  • Evaluations & Prompt Optimization: LLM-as-judge Evaluators
  • Evaluations & Prompt Optimization: Build your own Eval
  • Data Flywheel: Datasets
  • Data Flywheel: Auto-Building Datasets (from real-time trace filters and automated LLM evaluations)
  • Data Flywheel: User Feedback Collection
  • Data Flywheel: Human Annotation
  • Data Flywheel: Human Annotation Queues
  • Data Flywheel: Automated LLM-as-a-judge Augmenting
  • Data Flywheel: Export all Traces and Data
  • LangWatch Safeguards: Jailbreaking / Prompt Injection
  • LangWatch Safeguards: Business Sensitive evaluation
  • LangWatch Safeguards: PII detection and auto-redaction
  • LangWatch Safeguards: Competitor blocklist, off-topic evaluation
  • LangWatch Safeguards: Content Moderation
  • LangWatch Safeguards: Custom Guardrails
  • LangWatch Analytics: User-Analytics, Topic Detection, Sentiment Analysis, Feedback
  • LangWatch Analytics: Customizable graphs on any metric available in the platform
  • LangWatch Analytics: Tracking functional KPIs, allowing stakeholders to visualize performance metrics in real time
  • LangWatch Analytics: Trend analysis and performance benchmarking
  • LangWatch Analytics: Detailed tracking of costs including per-request costs and overall operational expenses
  • Collaboration: Projects: Unlimited
  • Collaboration: Users: Unlimited
  • API: Extensive Public API
  • Support: Community (GitHub, Discord)
  • Support: Chat & Email
  • Support: Private Slack/Discord Channel
  • Support: Dedicated Support Engineer
  • Support: Support SLA
  • Support: Architectural Guidance
  • SSO via Google, AzureAD, GitHub (Microsoft)
  • RBAC
  • Enterprise SSO (Okta, AzureAD/EntraID)
  • Scale-up add-on
  • SSO Enforcement
  • Data Retention Management
  • Audit Logs
  • Billing & Compliance: Subscription Management: Account Manager
  • Billing & Compliance: Payment Methods: Custom
  • Billing & Compliance: Contract Duration: Custom
  • Billing & Compliance: Billing via AWS Marketplace
  • Billing & Compliance: Scale-up add-on
  • Billing & Compliance: Contracts: Custom
  • Billing & Compliance: Data Processing Agreement (GDPR)
  • Billing & Compliance: ISO27001 Reports

Details

Pricing Tier

Free

Categories

Developer ToolsData Analysis

Target Audience

Startups

Integrations

Source &Integrates seamlessly with existing testing infrastructure and observability stacks.Works with any LLM application, agent framework, or model, offering broad compatibility.Ensures no data lock-in, allowing easy export and interoperation with your existing stack.HostableA fully open-source platform, deployable locally or self-hosted for complete control.

Sponsor

Ad space