Gradio

Comparing Large Language Models in an open and reproducible way

Last updated: June 25, 2024 at 10:30 AM


10	🔴 CLOSED	togethercomputer/RedPajama-INCITE-7B-Instruct	86.48	78.49	58.47	72.29	58.22	21.19	80.17	54.39 kg


1	🟢 OPEN	01-ai/Yi-1.5-9B-Chat	86.48	78.49	68.5	72.29	58.22	21.19	80.17	54.39 kg
2	🟢 OPEN	mosaicml/mpt-7b-instruct	85.38	44.96	58.47	65.33	23.73	14.62	53.04	16.96 kg
3	🟢 OPEN	togethercomputer/RedPajama-INCITE-7B-Instruct	84.36	58.33	45.38	62.79	55.65	43.68	74.69	49.89 kg
4	🟢 OPEN	stabilityai/stable-beluga-7b	83.16	67.4	54.89	23.81	28.66	29.51	72.83	64.12 kg
5	🟢 OPEN	mistralai/Mistral-7B-Instruct-v0.3	70.06	72.48	40.93	78.19	59.95	22.74	48.18	19.16 kg
6	🟢 OPEN	01-ai/Yi-1.5-34B-Chat	67.34	41.09	83.63	66.51	66.37	63.69	66.91	92.27 kg
7	🔴 CLOSED	databricks/dbrx-instruct	66.14	55.65	41.14	26.47	11.89	48.18	54.15	51.35 kg
8	🟢 OPEN	Qwen/Qwen2-7B-Instruct	62.8	77.11	48.99	50.85	45.54	12.79	67.34	17.88 kg
9	🟢 OPEN	meta-llama/llama-3-70b-instruct	58.73	87.04	72.94	55.92	19.36	19.36	42.61	86.75 kg
10	🟢 OPEN	google/gemma-7b-it	55.21	61.49	59.44	37.47	46.71	18.37	53.15	37.27 kg
11	🟢 OPEN	microsoft/phi-3-mini-4k-instruct	54.05	62.56	46.34	68.13	14.47	69.21	74.75	20.67 kg
12	🔴 CLOSED	OpenAI/gpt-4o	46.1	59.71	41.55	74.56	25.53	49.75	54.03	52.49 kg
13	🟢 OPEN	google/gemma-2b	44.42	41.76	42.04	39.52	33.32	26.28	77.29	36.32 kg
14	🔴 CLOSED	anthropic/claude-3-opus	43.25	86.93	83.45	68.5	28.28	15.86	70.79	44.58 kg
15	🟢 OPEN	microsoft/phi-3-mini-128k-instruct	40.28	78.93	71.81	63.74	56.28	14.44	56.13	12.47 kg