Github

Chat with Founders

x-api-key

Parea AI

API Reference

Blog

HomePage

Support

Haiku > GPT-4 Turbo > Opus >>> GPT-3.5 Turbo if not using parallel function calling.

Anthropic's Haiku Beats GPT-4 Turbo in Tool Use - Sometimes

What is Parea AI?

Evaluation Quickstart

Monitor your LLM requests and application functions.

Monitoring Quickstart

Evaluation

Overview

Record discrete events or related events in your application from LLM requests/chains to functions.

Logging and Tracing

Enrich events with context and details to facilitate analysis and debugging.

Metadata

Images

Record feedback from users on the quality LLM results.

Feedback

Use details from a trace log to build datasets.

Dataset from Trace

From Trace to Playground

Organize logs, traces & experiments into projects.

Projects

Attach evaluations to a trace to identify failure cases

Evaluations in Trace

OpenAI auto-logging also works with the new OpenAI Assistants Python API.

OpenAI Assistant

Using templated messages in auto-traced OpenAI & Anthropic calls

Templated LLM Calls

Manual Review

Compare

Add functions to prompts in the playground.

Function calling

Datasets

Add evaluation metrics to the playground.

Evaluation metrics

Evaluate your prompts on a dataset with evaluation metrics.

Trigger experiments

Deploy versioned prompts and use them via the SDK.

Deployed Prompts

Instrumenting your Langchain applications with Parea AI

LangChain

Instrument & test `instructor` code with Parea AI

Instructor

Instrumenting your DSPy application with Parea AI

DSPy

Instrumenting your SGLang application with Parea AI

SGLang

Instrumenting your OpenAI calls made through Trigger.dev with Parea

Trigger.dev

On-premise deployment of Parea in your environment

Self-hosting

Tutorial on improving a Langchain RAG application using Parea's Evals, Tracing, and Playground.

Optimize a LangChain RAG App

How many samples are necessary to achieve good performance with DSPy?

Optimize a RAG DSPy Application

You will need an API Key and Organization to use the API/SDK.

Authentication

Get a completion response using either one of your organization's deployed prompts, or by providing completion details including prompt and inputs in the request.

Completion

Get Deployed Prompt

Log a (LLM) response to visualize inference results, or chains.

Create Dataset

Get Dataset

Add Test Cases

Record Feedback

Lists experiments given a set of filters.

List Experiments

Fetch Experiment Logs

Python

TypeScript

How to measure the performance of retrieval applications without ground truth data.

Welcome

Guides

Tutorials