_mergeRanks is mutated as part of encoding: https://github.com/dotnet/machinelearning/blob/4635a862ddd21b3e7de0404f73a897fecb2011a1/src/Microsoft.ML.Tokenizers/Model/EnglishRoberta.cs#L591
Activity
stephentoub commentedon Feb 16, 2024
cc: @tarekgh