Skip to content

The mlm and mlm_probability arguments in the run.py are not effective. #173

Open
Open
The mlm and mlm_probability arguments in the run.py are not effective.#173

Description

@cridin1
cridin1
opened on Dec 2, 2023

If I want to pretrain a model on a MLM task, how can these args be used effectively?

parser.add_argument("--mlm", action='store_true',help="Train with masked-language modeling loss instead of language modeling.")
parser.add_argument("--mlm_probability", type=float, default=0.15,help="Ratio of tokens to mask for masked language modeling loss")

Activity

  1. cridin1
    changed the title [-]The mlm and mlm_probability arguments in all the run.py are not effective.[/-] [+]The mlm and mlm_probability arguments in the run.py are not effective.[/+] on Dec 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions