PalmX 2025 - Submission Rules

Rules

Participants can only fine-tune large language models (LLMs) that have fewer than 13 billion parameters.

Participants are permitted to fine-tune their LLMs on any external data.

Fine-tuned models must be shared with the organizers by July 25, 2025. Please see the instructions below on how to share your model with us.

The final submission for the test set will be generated using the following code:

https://colab.research.google.com/drive/1udVUMqMJKkCL90J_VG2vYDgj7bUBnV-J?usp=sharing

Dev Phase Submission

We have provided a development set for participants in both subtasks to build and evaluate their models locally. Participants can use the following code to evaluate their models:

https://colab.research.google.com/drive/1udVUMqMJKkCL90J_VG2vYDgj7bUBnV-J?usp=sharing

Test Phase Submission

The test data for the PalmX shared task will not be disclosed to participants.

Participants will need to submit their best model checkpoint by July 25, 2025. We will then evaluate the submissions locally on our test set and publish a public leaderboard on July 30, 2025.

How to Privately Share Your Model Checkpoints

To share your model with us, please follow these steps to ensure it remains private:

1

Upload your model to Hugging Face

You can find instructions on how to do this here: https://huggingface.co/docs/transformers/main/en/model_sharing

Make sure to set the model's visibility to "Private".

2

Create a fine-grained access token for your private model

This will allow us to access your model without you having to share your main account token.

Navigate to the "Create a new fine-grained access token" page on Hugging Face:

https://huggingface.co/settings/tokens/new?tokenType=fineGrained

Hugging Face Create Access Token Form showing Fine-grained token type selected, token name field, and repository permissions section

Fill out the token creation form as shown above

Give the token a descriptive name (e.g., "Palmx2025 Submission")

Under "Repository access," select "Only select repositories" and choose the private model you wish to share

Set the permission level to "Read"

Hugging Face token creation form showing repository permissions section with model selected and Read permission level highlighted

Configure repository permissions as shown - select your model and set to "Read" access

Click "Generate token"

Generated Access Token modal showing 'Save your Access Token' with token value and copy button

Important: Save your access token securely - you won't be able to see it again!

3

Share the access token with us

Copy the generated access token and email it to palmx2025@gmail.com.

Important: In your email, please be sure to include your team name and the subtask you participated in.