Discussions

• 2 You must be logged in to vote

  💬

  Paper writing

  Muennighoff started Apr 29, 2024 in General

  

  

  

  

  59
• 1 You must be logged in to vote

  💡

  Tracking CO2 Emissions

  x-tabdeveloping started May 14, 2024 in Ideas

  

  

  

  

  22
• 5 You must be logged in to vote

  💬

  Running models

  imenelydiaker started May 14, 2024 in General

  

  

  

  

  32
• 1 You must be logged in to vote

  💬

  Paper writing - Description of abstention metrics

  hgissbkh started Jun 10, 2024 in General

  

  

  4
• 1 You must be logged in to vote

  💬

  Sampling MTEB

  NehaB18 started May 8, 2024 in General

  

  

  2
• 1 You must be logged in to vote

  💬

  Experiments for the paper

  imenelydiaker started May 27, 2024 in General

  

  

  

  

  3
• 4 You must be logged in to vote

  💬

  Score Aggregation for Multilingual benchmark (and in general)

  KennethEnevoldsen started May 17, 2024 in General

  

  

  

  

  6
• 1 You must be logged in to vote

  💬

  Stratification of multilabel data

  dokato started May 14, 2024 in General

  

  

  7
• 1 You must be logged in to vote

  💬

  Ideas on Conversational Retrieval Task Category

  vaibhavad started May 10, 2024 in General

  

  

  

  

  7
• 2 You must be logged in to vote

  💬

  Table to count number of tasks per language

  isaac-chung started May 13, 2024 in General

  

  

  

  

  15
• 3 You must be logged in to vote

  💬

  Leaderboard changes for massive multilinguality

  Muennighoff started May 12, 2024 in General

  

  

  

  

  6
• 2 You must be logged in to vote

  💬

  Retrieval speed up

  isaac-chung started May 6, 2024 in General

  

  

  

  

  12
• 1 You must be logged in to vote

  💡

  Linguistic Probing and New Task

  asparius started Apr 26, 2024 in Ideas

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  Retrieval evaluator with 0 scores

  sroy-forr asked Apr 28, 2024 in Q&A · Answered

  

  

  3
• 2 You must be logged in to vote

  💬

  Datasets of language identification

  imenelydiaker started Apr 29, 2024 in General

  

  

  

  

  10
• 1 You must be logged in to vote

  🙏

  Required dataformat for corpus.jsonl, queries.jsonl, test.tsv

  paulesause asked Apr 26, 2024 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  💬

  Should we include Cross-Lingual tasks in MMTEB?

  orionw started Apr 13, 2024 in General

  

  

  

  

  13
• 1 You must be logged in to vote

  🙏

  Clarification for model scores

  shreeya-dhakal asked Apr 23, 2024 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Clarifications regarding datasets and task

  shreeya-dhakal asked Apr 20, 2024 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  🙏

  Minimum rows for dataset

  sroy-forr asked Apr 19, 2024 in Q&A · Answered

  

  

  4
• 1 You must be logged in to vote

  🙏

  Adding a dataset on hugging face?

  Sakshamrzt asked Apr 7, 2024 in Q&A · Answered

  

  

  

  

  8
• 1 You must be logged in to vote

  💬

  Threads for Twitter and LinkedIn for MMTEB

  imenelydiaker started Apr 4, 2024 in General

  

  

  

  

  17
• 1 You must be logged in to vote

  💬

  Registring models and results

  imenelydiaker started Apr 4, 2024 in General

  

  

  

  

  3
• 1 You must be logged in to vote

  💬

  Scaling the benchmark

  imenelydiaker started Apr 4, 2024 in General

  

  

  

  

  5
• 1 You must be logged in to vote

  🙌

  Running 7b models on the benchmark

  wendlerc started Mar 19, 2024 in Show and tell

  

  

  

  

  21