Auto Classes ​

Auto classes can help you automatically retrieve relevant models based on given names or paths of pretrained weights, configurations, and vocabularies.

Instantiating AutoConfig, AutoModel, or AutoTokenizer by passing the name or path of a pretrained model will directly create a relevant model architecture. An example code snippet is as follows:

from openmind import AutoModel

model = AutoModel.from_pretrained("PyTorch-NPU/bert_base_cased")

A model will be created, which is an instance of BertModel.

In addition, each task has an AutoModel class for each backend framework (PyTorch or MindSpore). The following table lists the frameworks supported by each API class.

Extending Auto Classes ​

Each auto class has a method available for extension with custom classes. For example, if you have defined a custom model class called NewModel, just ensure that you have a NewModelConfig and then you can add them to the auto classes.

from openmind import AutoConfig, AutoModel

AutoConfig.register("new-model", NewModelConfig)
AutoModel.register(NewModelConfig, NewModel)

Then, you can use auto classes by passing in the name or path of the pretrained model.

General ​

openmind.AutoConfig Class ​

Function ​

This is a generic configuration class. When created using the AutoConfig.from_pretrained class method, it will be instantiated as a registered configuration class.

APIs ​

from_pretrained Method ​

from_pretrained(pretrained_model_name_or_path, **kwargs): Instantiates a configuration class in the library from a pretrained model configuration. The configuration class to be instantiated is selected based on the model_type attribute of the loaded configuration object, or when it is missing, by rolling back to match pretrained_model_name_or_path.

Parameters:

• pretrained_model_name_or_path (str or os.PathLike):

  • Character string, the model id of a pretrained model in openMind Models.
  • Path to a directory that contains a configuration file, which is saved using the save_pretrained(), for example, ./my_model_directory/.
  • Path or URL to a saved configuration.json file, for example, ./my_model_directory/configuration.json.

• yaml_name_or_path (str):

  • Only MindSpore is supported. If pretrained_model_name_or_path is set, yaml_name_or_path becomes invalid.
  • Character string, which is a supported model name or the path of a model configuration file (.yaml).

• cache_dir (str or os.PathLike, optional): Path in which a downloaded pretrained model configuration should be cached if the default cache path is not used.

• force_download (bool, optional, defaults to False): Whether to forcibly (re-)download model weights and configuration files and overwrite the existing cached versions.

• resume_download (bool, optional, defaults to False): Whether to resume files that are not completely received before the download.

• proxies (Dict[str, str], optional): Proxy server dictionary configured in compliance with the protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. If it is set, the proxy will be used on every request.

• revision (str, optional, defaults to "main"): Specific model version to be used. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• return_unused_kwargs (bool, optional, defaults to False): If the value is False, this function returns only the final configuration object.

  If the value is True, this function returns a Tuple(config, unused_kwargs), where unused_kwargs is a dictionary consisting of key-value pairs, and the keys are not configuration attributes.

• trust_remote_code (bool, optional, defaults to False): Whether custom models are allowed to be defined in model files within model repositories. This option should only be set to True if you trust the model repositories and have read their code, as it will execute the code on your local machine.

• kwargs (additional keyword arguments, optional): In kwargs, the values of any keys which are configuration attributes will be used to overwrite the corresponding configuration attributes. If the keys are not configuration attributes, they are controlled by the return_unused_kwargs keyword parameter.

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

  • token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

Example:

from openmind import AutoConfig

# Download a configuration from openMind Models.
config = AutoConfig.from_pretrained("PyTorch-NPU/bert_base_uncased")

# If the configuration is saved using save_pretrained('./test/saved_model/'), you can load it as follows:
config = AutoConfig.from_pretrained("./test/bert_saved_model/")

# If a specific configuration file is saved using save_pretrained('./test/saved_model/'), you can load it as follows:
config = AutoConfig.from_pretrained("./test/bert_saved_model/my_configuration.json")

# Change the config attributes when loading a pretrained config.
config = AutoConfig.from_pretrained("PyTorch-NPU/bert_base_uncased", output_attentions=True, foo=False)
config.output_attentions
'''
Output:
True
'''

config, unused_kwargs = AutoConfig.from_pretrained("PyTorch-NPU/bert_base_uncased", output_attentions=True, foo=False, return_unused_kwargs=True)
config.output_attentions
'''
Output:
True
'''

unused_kwargs
'''
Output:
{'foo': False}
'''

register Method ​

register(model_type, config, exist_ok=False): Registers a new configuration for this class.

Parameters

• model_type (str): Model type, for example, BERT or GPT2.
• config (PretrainedConfig): Configuration file to be registered.

Constraint ​

This class cannot be directly instantiated using __init__(). Otherwise, an error will be thrown.

openmind.AutoTokenizer Class ​

Function ​

This is a generic tokenizer class. When created using the AutoTokenizer.from_pretrained class method, it will be instantiated as a registered tokenizer class.

APIs ​

from_pretrained Method ​

from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs): Instantiates a tokenizer class in the library from a pretrained model vocabulary. The tokenizer class to be instantiated is selected based on the model_type attribute of the configuration object (which can be passed as an argument or loaded from pretrained_model_name_or_path), or when it is missing, by rolling back to match pretrained_model_name_or_path.

Parameters

• pretrained_model_name_or_path (str or os.PathLike):

  • Character string, the model id of a pretrained model in openMind Models.
  • Path to a directory that contains a vocabulary file required by the tokenizer, which is saved using the save_pretrained() method, for example, ./my_model_directory/.
  • Path or URL to a single saved vocabulary file if and only if the tokenizer requires only a single vocabulary file (like BERT or XLNet), for example, ./my_model_directory/vocab.txt. (Not applicable to all derived classes)

• yaml_name_or_path (str):

  • Only MindSpore is supported. If pretrained_model_name_or_path is set, yaml_name_or_path becomes invalid.
  • Character string, which is a supported model name or the path of a model configuration file (.yaml).

• inputs (additional location parameter, optional): Will be passed to the Tokenizer.__init__() method.

• config ([PretrainedConfig], optional): Identifies the configuration object of the tokenizer class to be instantiated.

• cache_dir (str or os.PathLike, optional): Path in which a downloaded pretrained model configuration should be cached if the default cache path is not used.

• force_download (bool, optional, defaults to False): Whether to forcibly (re-)download model weights and configuration files and overwrite the existing cached versions.

• resume_download (bool, optional, defaults to False): Whether to resume files that are not completely received before the download.

• proxies (Dict[str, str], optional): Proxy server dictionary used by protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. If it is set, the proxy will be used on every request.

• revision (str, optional, defaults to "main"): Specific model version to be used. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• subfolder (str, optional): If relevant files are located in a subfolder within a model repository on openMind, specify the subfolder here.

• use_fast (bool, optional, defaults to True): Use a fast Rust-based tokenizer if a given model supports it. If the given model does not support it, a Python-based tokenizer is returned.

• tokenizer_type (str, optional): Type of a loaded tokenizer.

• trust_remote_code (bool, optional, defaults to False): Whether custom models are allowed to be defined in model files within model repositories. This option should only be set to True if you trust the model repositories and have read their code, as it will execute the code on your local machine.

• kwargs (additional keyword arguments, optional): Will be passed to the Tokenizer.__init__() method. It can be used to set special tokens, such as bos_token, eos_token, unk_token, sep_token, pad_token, cls_token, mask_token, and additional_special_tokens. For more details, see parameters in __init__().

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

  • token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

Example:

from openmind import AutoTokenizer

# Download a vocabulary from openMind Models.
tokenizer = AutoTokenizer.from_pretrained("PyTorch-NPU/bert_base_uncased")

# If the tokenizer is saved using save_pretrained('./test/saved_model/') and the vocabulary file is in the folder, you can load it as follows:
tokenizer = AutoTokenizer.from_pretrained("./test/bert_saved_model/")

register Method ​

register(config_class, slow_tokenizer_class=None, fast_tokenizer_class=None, exist_ok=False): Registers a new tokenizer.

Parameters

• config_class (str): Configuration corresponding to the model to be registered.
• slow_tokenizer_class (PretrainedTokenizer, optional): Slow tokenizer to be registered.
• fast_tokenizer_class (PretrainedTokenizerFast, optional): Fast tokenizer to be registered.

Constraint ​

This class cannot be directly instantiated using __init__(). Otherwise, an error will be thrown.

openmind.AutoProcessor Class ​

Function ​

This is a generic processor class. When created using the AutoProcessor.from_pretrained class method, it will be instantiated as a registered processor class. This class cannot be directly instantiated using __init__(). Otherwise, an error will be thrown.

APIs ​

from_pretrained Method ​

from_pretrained(pretrained_model_name_or_path, **kwargs): Instantiates a processor class in the library from a pretrained model vocabulary. The processor class to be instantiated is selected based on the model_type attribute of the configuration object (which can be passed as an argument or loaded from pretrained_model_name_or_path).

Parameters

• pretrained_model_name_or_path (str or os.PathLike):

  • Character string, the model id of a pretrained model in openMind Models.
  • Path to a directory that contains a processor file, which is saved using the save_pretrained() method, for example, ./my_model_directory/.

• yaml_name_or_path (str):

  • Only MindSpore is supported. If pretrained_model_name_or_path is set, yaml_name_or_path becomes invalid.
  • Character string, which is a supported model name or the path of a model configuration file (.yaml).

• cache_dir (str or os.PathLike, optional): Path in which the feature extractor of a downloaded pretrained model should be cached if the default cache path is not used.

• force_download (bool, optional, defaults to False): Whether to forcibly (re-)download feature extractor files and overwrite the existing cached versions.

• resume_download (bool, optional, defaults to False): Whether to resume files that are not completely received before the download.

• proxies (Dict[str, str], optional): Proxy server dictionary used by protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. If it is set, the proxy will be used on every request.

• token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

• revision (str, optional, defaults to "main"): Specific model version to be used. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• return_unused_kwargs (bool, optional, defaults to False): If the value is False, this function returns only the final feature extractor object. If the value is True, this function returns a tuple (feature_extractor, unused_kwargs). unused_kwargs is a dictionary consisting of key-value pairs, whose keys are not feature extractor attributes, and they are not used to update the feature extractor and are ignored in kwargs.

• trust_remote_code (bool, optional, defaults to False): Whether custom models are allowed to be defined in model files within model repositories. This option should only be set to True if you trust the model repositories and have read their code, as it will execute the code on your local machine.

• kwargs (additional keyword arguments, optional): In kwargs, the values of any keys which are configuration attributes will be used to overwrite the corresponding configuration attributes. If the keys are not image processor attributes, they are controlled by the return_unused_kwargs keyword parameter.

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

register Method ​

register(config_class, processor_class, exist_ok=False): Registers a new processor.

Parameters

• config_class (str): Configuration corresponding to the model to be registered.

• processor_class (FeatureExtractorMixin): Processor to be registered.

openmind.AutoImageProcessor Class ​

Description ​

This is a generic image processor class. When created using the AutoImageProcessor.from_pretrained class method, it will be instantiated as a registered image processor class.

APIs ​

from_pretrained Method ​

from_pretrained(pretrained_model_name_or_path, **kwargs): Instantiates an image processor class in the library from a pretrained model vocabulary. The image processor class to be instantiated is selected based on the model_type attribute of the configuration object (which can be passed as an argument or loaded from pretrained_model_name_or_path), or when it is missing, by rolling back to match pretrained_model_name_or_path.

Parameters

• pretrained_model_name_or_path (str or os.PathLike):

  • Character string, the model id of a pretrained model in openMind Models.
  • Path to a directory that contains an image processor file, which is saved using the save_pretrained() method, for example, ./my_model_directory/.
  • Path or URL to a saved image processor JSON file, for example, ./my_model_directory/preprocessor_config.json.

• cache_dir (str or os.PathLike, optional): Path in which the image processor of a downloaded pretrained model should be cached if the default cache path is not used.

• force_download (bool, optional, defaults to False): Whether to forcibly (re-)download image processor files and overwrite the existing cached versions.

• resume_download (bool, optional, defaults to False): Whether to resume files that are not completely received before the download.

• proxies (Dict[str, str], optional): Proxy server dictionary used by protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. If it is set, the proxy will be used on every request.

• token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

• revision (str, optional, defaults to "main"): Specific model version to be used. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• return_unused_kwargs (bool, optional, defaults to False): If the value is False, this function returns only the final image processor object. If the value is True, this function returns a tuple (image processor, unused_kwargs). unused_kwargs is a dictionary consisting of key-value pairs, whose keys are not image processor attributes, and they are not used to update the image processor and are ignored in kwargs.

• trust_remote_code (bool, optional, defaults to False): Whether custom models are allowed to be defined in model files within model repositories. This option should only be set to True if you trust the model repositories and have read their code, as it will execute the code on your local machine.

• kwargs (additional keyword arguments, optional): In kwargs, the values of any keys which are image processor attributes will be used to overwrite the corresponding image processor attributes. If the keys are not image processor attributes, they are controlled by the return_unused_kwargs keyword parameter.

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

register Method ​

register(config_class, image_processor_class, exist_ok=False): Registers a new image processor.

Parameters

• config_class (str): Configuration corresponding to the model to be registered.

• image_processor_class (FeatureExtractorMixin): Image processor to be registered.

Constraint ​

This class cannot be directly instantiated using __init__(). Otherwise, an error will be thrown.

openmind.AutoFeatureExtractor Class (PyTorch) ​

MindSpore does not support this type.

Description ​

This is a generic feature extractor class. When created using the AutoFeatureExtractor.from_pretrained class method, it will be instantiated as a registered feature extractor class. This class cannot be directly instantiated using __init__(). Otherwise, an error will be thrown.

APIs ​

from_pretrained Method ​

from_pretrained(pretrained_model_name_or_path, **kwargs): Instantiates a feature extractor class in the library from a pretrained model vocabulary. The feature extractor class to be instantiated is selected based on the model_type attribute of the configuration object (which can be passed as an argument or loaded from pretrained_model_name_or_path), or when it is missing, by rolling back to match pretrained_model_name_or_path.

Parameters

• pretrained_model_name_or_path (str or os.PathLike):

  • Character string, the model id of a pretrained model in openMind Models.
  • Path to a directory that contains a feature extractor file, which is saved using the save_pretrained(), for example, ./my_model_directory/.
  • Path or URL to a saved feature extractor JSON file, for example, ./my_model_directory/preprocessor_config.json.

• cache_dir (str or os.PathLike, optional): Path in which the feature extractor of a downloaded pretrained model should be cached if the default cache path is not used.

• force_download (bool, optional, defaults to False): Whether to forcibly (re-)download feature extractor files and overwrite the existing cached versions.

• resume_download (bool, optional, defaults to False): Whether to resume files that are not completely received before the download.

• proxies (Dict[str, str], optional): Proxy server dictionary used by protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. If it is set, the proxy will be used on every request.

• token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

• revision (str, optional, defaults to "main"): Specific model version to be used. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• return_unused_kwargs (bool, optional, defaults to False):

  If the value is False, this function returns only the final feature extractor object. If the value is True, this function returns a tuple (feature_extractor, unused_kwargs). unused_kwargs is a dictionary consisting of key-value pairs, whose keys are not feature extractor attributes, and they are not used to update the feature extractor and are ignored in kwargs.

• trust_remote_code (bool, optional, defaults to False): Whether custom models are allowed to be defined in model files within model repositories. This option should only be set to True if you trust the model repositories and have read their code, as it will execute the code on your local machine.

• kwargs (additional keyword arguments, optional): In kwargs, the values of any keys which are feature extractor attributes will be used to overwrite the corresponding feature extractor attributes. If the keys are not feature extractor attributes, they are controlled by the return_unused_kwargs keyword parameter.

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

register Method ​

register(config_class, feature_extractor_class, exist_ok=False): Registers a new feature extractor.

Parameters

• config_class (str): Configuration corresponding to the model to be registered.

• feature_extractor_class (FeatureExtractorMixin): Feature extractor to be registered.

Generic Model Classes ​

The following auto classes can be used to instantiate a base model class without specifying a model head.

openmind.AutoModel Class ​

Function ​

This is a generic model class. When created using the from_pretrained() or from_config() method, it will be instantiated as a registered base model class. This class cannot be directly instantiated using __init__(). Otherwise, an error will be thrown.

APIs ​

from_config Method ​

from_config(**kwargs): Instantiates a base model class in the library from a configuration. Note that loading a model from its configuration file does not load the model weights. It affects only the configuration of the model. You can use from_pretrained() to load the model weights.

Parameters

• config (PretrainedConfig): The model class to be instantiated is selected based on the registered configuration class.

• attn_implementation (str, optional): Only PyTorch is supported. It indicates the attention implementation to be used in the model (if relevant). The value can be eager (custom attention), sdpa (using F.scaled_dot_product_attention), or flash_attention_2. By default, SDPA will be used for torch>=2.1.1 if available. Otherwise, the custom eager implementation is used by default.

• kwargs (additional keyword arguments, optional): In kwargs, the values of any keys which are model attributes will be used to overwrite the corresponding model attributes.

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

  • token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

  • download_checkpoint (bool, optional, defaults to True): Only MindSpore is supported. (Only origin_mode is supported.) Indicates whether to download the checkpoint file of a model.

Example:

from openmind import AutoConfig, AutoModel

config = AutoConfig.from_pretrained("PyTorch-NPU/bert_base_cased")
model = AutoModel.from_config(config)

from_pretrained Method ​

from_pretrained(*model_args, **kwargs): Instantiates a base model class in the library from a pretrained model. The model class to be instantiated is selected based on the model_type attribute of the configuration object (which can be passed as an argument or loaded from pretrained_model_name_or_path), or when it is missing, by rolling back to match pretrained_model_name_or_path.

Parameters

• pretrained_model_name_or_path (str or os.PathLike):

  • Character string, the model id of a pretrained model hosted in openMind Models.
  • Path to a directory that contains a model weight file, which is saved using the save_pretrained() method, for example, ./my_model_directory/.

• model_args (additional location parameter, optional): Will be passed to the __init__() method of the base model.

• config (PretrainedConfig, optional): It is used to replace the automatically loaded configuration. The configuration can be automatically loaded in the following situations:

  • The model is provided by the library (loaded using the model id field of a pretrained model).
  • The model is saved using save_pretrained() and is reloaded through the saved directory.
  • The model provides a local directory as pretrained_model_name_or_path, and a config.json file exists in the directory.

• state_dict (Dict[str, torch.Tensor], optional): State dictionary to use instead of the one loaded from the saved weight file.

  If you want to create a model using a pretrained configuration but load your own weights, you can use this option.

• cache_dir (str or os.PathLike, optional): Path in which a downloaded pretrained model configuration should be cached if the default cache path is not used.

• from_tf (bool, optional, defaults to False) (not supported by openMind): Loads the weight from the checkpoint file of TensorFlow (For details, see pretrained_model_name_or_path.).

• force_download (bool, optional, defaults to False): Whether to forcibly (re-)download feature extractor files and overwrite the existing cached versions.

• resume_download (bool, optional, defaults to False): Whether to resume files that are not completely received before the download.

• proxies (Dict[str, str], optional): Proxy server dictionary configured in compliance with the protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. If it is set, the proxy will be used on every request.

• output_loading_info (bool, optional, defaults to False): Whether to return a dictionary containing missing keys, unexpected keys, and error messages.

• local_files_only (bool, optional, defaults to False): Whether to view only local files (for example, without attempting to download models).

• revision (str, optional, defaults to "main"): Specific model version to be used. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• trust_remote_code (bool, optional, defaults to False): Whether custom models are allowed to be defined in model files within model repositories. This option should only be set to True if you trust the model repositories and have read their code, as it will execute the code on your local machine.

• code_revision (str, optional, defaults to "main"): A specific version of code in the model library if the code is in a different repository from the rest of the model. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• kwargs (additional keyword arguments, optional): Used to update the loaded configuration object and initialize the model (for example, output_attentions=True). Its behavior depends on whether config is provided or automatically loaded:

  • If a configuration is provided with config, kwargs will be directly passed to the __init__ method of the underlying model (assuming that all related updates to the configuration have been completed).

  • If no configuration is provided, kwargs will be first passed to the initialization function (from_pretrained()) of the configuration class. The value of each key in kwargs will overwrite the attribute corresponding to the key in the configuration. The remaining keys that do not correspond to any configuration attribute will be passed to the __init__ function of the underlying model.

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

  • token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

  • ignore_mismatched_sizes (bool, optional, defaults to False): Only MindSpore is supported. ChWhether to throw errors when some weights at the checkpoint are different from the weights of the model (for example, when a checkpoint containing three tags is used to instantiate a model containing 10 tags).

  • variant (str, optional): Only MindSpore is supported. If the variant file name is specified, the weight is loaded from the variant file name.

  • download_checkpoint (bool, optional, defaults to True): Only MindSpore is supported. (Only origin_mode is supported.) Indicates whether to download the checkpoint file of a model.

Example:

from openmind import AutoConfig, AutoModel

# Download a model and a configuration from openMind Models.
model = AutoModel.from_pretrained("PyTorch-NPU/bert_base_cased")

# Update the configuration during loading.
model = AutoModel.from_pretrained("PyTorch-NPU/bert_base_cased", output_attentions=True)
model.config.output_attentions
'''
Output:
True
'''

NLP ​

The following auto classes can be used for the following NLP tasks.

openmind.AutoModelForCausalLM Class ​

Description ​

This is a generic model class. When created using the from_pretrained() or from_config() method, it will be instantiated as a registered model class (with a causal language model head). This class cannot be directly instantiated using __init__(). Otherwise, an error will be thrown.

APIs ​

from_config Method ​

from_config(**kwargs): Instantiates a base model class in the library from a configuration. Note that loading a model from its configuration file does not load the model weights. It affects only the configuration of the model. You can use from_pretrained() to load the model weights.

Parameters

• config (PretrainedConfig): The model class to be instantiated is selected based on the registered configuration class.

• attn_implementation (str, optional): Only PyTorch is supported. It indicates the attention implementation to be used in the model (if relevant). The value can be eager (custom attention), sdpa (using F.scaled_dot_product_attention), or flash_attention_2. By default, SDPA will be used for torch>=2.1.1 if available. Otherwise, the custom eager implementation is used by default.

• kwargs (additional keyword arguments, optional): In kwargs, the values of any keys which are model attributes will be used to overwrite the corresponding model attributes.

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

  • token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

  • download_checkpoint (bool, optional, defaults to True): Only MindSpore is supported. (Only origin_mode is supported.) Indicates whether to download the checkpoint file of a model.

Example:

from openmind import AutoConfig, AutoModelForCausalLM

# Download a configuration from openMind Models.
config  = AutoConfig.from_pretrained("PyTorch-NPU/bert_base_cased")
model = AutoModelForCausalLM.from_config(config)

from_pretrained Method ​

from_pretrained(*model_args, **kwargs): Instantiates a model class (with a causal language model head) in the library from a pretrained model. The model class to be instantiated is selected based on the model_type attribute of the configuration object (which can be passed as an argument or loaded from pretrained_model_name_or_path), or when it is missing, by rolling back to match pretrained_model_name_or_path.

Parameters

• pretrained_model_name_or_path (str or os.PathLike):

  • Character string, the model id of a pretrained model hosted in openMind Models.
  • Path to a directory that contains a model weight file, which is saved using the save_pretrained() method, for example, ./my_model_directory/.

• model_args (additional location parameter, optional): Will be passed to the __init__() method of the base model.

• config (PretrainedConfig, optional): It is used to replace the automatically loaded configuration. The configuration can be automatically loaded in the following situations:

  • The model is provided by the library (loaded using the model id field of a pretrained model).
  • The model is saved using save_pretrained() and is reloaded through the saved directory.
  • The model provides a local directory as pretrained_model_name_or_path, and a config.json file exists in the directory.

• state_dict (Dict[str, torch.Tensor], optional): State dictionary to use instead of the one loaded from the saved weight file.

  If you want to create a model using a pretrained configuration but load your own weights, you can use this option.

• cache_dir (str or os.PathLike, optional): Path in which a downloaded pretrained model configuration should be cached if the default cache path is not used.

• from_tf (bool, optional, defaults to False) (not supported by openMind): Loads the weight from the checkpoint file of TensorFlow (For details, see pretrained_model_name_or_path.).

• force_download (bool, optional, defaults to False): Whether to forcibly (re-)download feature extractor files and overwrite the existing cached versions.

• resume_download (bool, optional, defaults to False): Whether to resume files that are not completely received before the download.

• proxies (Dict[str, str], optional): Proxy server dictionary configured in compliance with the protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. If it is set, the proxy will be used on every request.

• output_loading_info (bool, optional, defaults to False): Whether to return a dictionary containing missing keys, unexpected keys, and error messages.

• local_files_only (bool, optional, defaults to False): Whether to view only local files (for example, without attempting to download models).

• revision (str, optional, defaults to "main"): Specific model version to be used. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• trust_remote_code (bool, optional, defaults to False): Whether custom models are allowed to be defined in model files within model repositories. This option should only be set to True if you trust the model repositories and have read their code, as it will execute the code on your local machine.

• code_revision (str, optional, defaults to "main"): A specific version of code in the model library if the code is in a different repository from the rest of the model. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• kwargs (additional keyword arguments, optional): Used to update the loaded configuration object and initialize the model (for example, output_attentions=True). Its behavior depends on whether config is provided or automatically loaded:

  • If a configuration is provided with config, kwargs will be directly passed to the __init__ method of the underlying model (assuming that all related updates to the configuration have been completed).

  • If no configuration is provided, kwargs will be first passed to the initialization function (from_pretrained()) of the configuration class. The value of each key in kwargs will overwrite the attribute corresponding to the key in the configuration. The remaining keys that do not correspond to any configuration attribute will be passed to the __init__ function of the underlying model.

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

  • token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

  • ignore_mismatched_sizes (bool, optional, defaults to False): Only MindSpore is supported. ChWhether to throw errors when some weights at the checkpoint are different from the weights of the model (for example, when a checkpoint containing three tags is used to instantiate a model containing 10 tags).

  • variant (str, optional): Only MindSpore is supported. If the variant file name is specified, the weight is loaded from the variant file name.

  • download_checkpoint (bool, optional, defaults to True): Only MindSpore is supported. (Only origin_mode is supported.) Indicates whether to download the checkpoint file of a model.

Example:

from openmind import AutoConfig, AutoModelForCausalLM

# Download a configuration from openMind Models.
model  = AutoModelForCausalLM.from_pretrained("PyTorch-NPU/bert_base_cased")

# Update the configuration during loading.
model = AutoModelForCausalLM.from_pretrained("PyTorch-NPU/bert_base_cased", output_attentions=True)
model.config.output_attentions
'''
Output:
True
'''

openmind.AutoModelForSequenceClassification Class ​

Description ​

This is a generic model class. When created using the from_config() or from_pretrained() method, it will be instantiated as a registered model class (with a sequence classification head). This class cannot be directly instantiated using __init__(). Otherwise, an error will be thrown.

APIs ​

from_config Method ​

from_config(**kwargs): Instantiates a base model class in the library from a configuration. Note that loading a model from its configuration file does not load the model weights. It affects only the configuration of the model. You can use from_pretrained() to load the model weights.

Parameters

• config (PretrainedConfig): The model class to be instantiated is selected based on the registered configuration class.

• attn_implementation (str, optional): Only PyTorch is supported. It indicates the attention implementation to be used in the model (if relevant). The value can be eager (custom attention), sdpa (using F.scaled_dot_product_attention), or flash_attention_2. By default, SDPA will be used for torch>=2.1.1 if available. Otherwise, the custom eager implementation is used by default.

• kwargs (additional keyword arguments, optional): In kwargs, the values of any keys which are model attributes will be used to overwrite the corresponding model attributes.

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

  • token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

  • download_checkpoint (bool, optional, defaults to True): Only MindSpore is supported. (Only origin_mode is supported.) Indicates whether to download the checkpoint file of a model.

Example:

from openmind import AutoConfig, AutoModelForSequenceClassification

# Download a configuration from openMind Models.
config  = AutoConfig.from_pretrained("PyTorch-NPU/bert_base_cased")
model = AutoModelForSequenceClassification.from_config(config)

from_pretrained Method ​

from_pretrained(*model_args, **kwargs): Instantiates a model class (with a sequence classification head) in the library from a pretrained model. The model class to be instantiated is selected based on the model_type attribute of the configuration object (which can be passed as an argument or loaded from pretrained_model_name_or_path), or when it is missing, by rolling back to match pretrained_model_name_or_path.

Parameters

• pretrained_model_name_or_path (str or os.PathLike):

  • Character string, the model id of a pretrained model hosted in openMind Models.
  • Path to a directory that contains a model weight file, which is saved using the save_pretrained() method, for example, ./my_model_directory/.

• model_args (additional location parameter, optional): Will be passed to the __init__() method of the base model.

• config (PretrainedConfig, optional): It is used to replace the automatically loaded configuration. The configuration can be automatically loaded in the following situations:

  • The model is provided by the library (loaded using the model id field of a pretrained model).
  • The model is saved using save_pretrained() and is reloaded through the saved directory.
  • The model provides a local directory as pretrained_model_name_or_path, and a config.json file exists in the directory.

• state_dict (Dict[str, torch.Tensor], optional): State dictionary to use instead of the one loaded from the saved weight file.

  If you want to create a model using a pretrained configuration but load your own weights, you can use this option. It is recommended that you check whether save_pretrained() and from_pretrained().

• cache_dir (str or os.PathLike, optional): Path in which a downloaded pretrained model configuration should be cached if the default cache path is not used.

• from_tf (bool, optional, defaults to False) (not supported by openMind): Loads the weight from the checkpoint file of TensorFlow (For details, see pretrained_model_name_or_path.).

• force_download (bool, optional, defaults to False): Whether to forcibly (re-)download feature extractor files and overwrite the existing cached versions.

• resume_download (bool, optional, defaults to False): Whether to resume files that are not completely received before the download.

• proxies (Dict[str, str], optional): Proxy server dictionary configured in compliance with the protocol or endpoint, for example, {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. If it is set, the proxy will be used on every request.

• output_loading_info (bool, optional, defaults to False): Whether to return a dictionary containing missing keys, unexpected keys, and error messages.

• local_files_only (bool, optional, defaults to False): Whether to view only local files (for example, without attempting to download models).

• revision (str, optional, defaults to "main"): Specific model version to be used. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• trust_remote_code (bool, optional, defaults to False): Whether custom models are allowed to be defined in model files within model repositories. This option should only be set to True if you trust the model repositories and have read their code, as it will execute the code on your local machine.

• code_revision (str, optional, defaults to "main"): A specific version of code in the model library if the code is in a different repository from the rest of the model. It can be a branch name, tag name, or commit ID. Because a Git-based system is used in openMind Library to store models and other files, revision can be any identifier allowed by Git.

• ignore_mismatched_sizes (bool, optional, defaults to False): Only MindSpore is supported. ChWhether to throw errors when some weights at the checkpoint are different from the weights of the model (for example, when a checkpoint containing three tags is used to instantiate a model containing 10 tags).

• kwargs (additional keyword arguments, optional): Used to update the loaded configuration object and initialize the model (for example, output_attentions=True). Its behavior depends on whether config is provided or automatically loaded:

  • If a configuration is provided with config, kwargs will be directly passed to the __init__ method of the underlying model (assuming that all related updates to the configuration have been completed).

  • If no configuration is provided, kwargs will be first passed to the initialization function (from_pretrained()) of the configuration class. The value of each key in kwargs will overwrite the attribute corresponding to the key in the configuration. The remaining keys that do not correspond to any configuration attribute will be passed to the __init__ function of the underlying model.

  • use_auth_token (str, optional) (to be deprecated): User access token required for accessing non-public data in Hub.

  • token (str, optional): User access token required for accessing non-public data in the Hub. (Either use_auth_token or token must be configured for accessing non-public data. Configuring token is recommended.)

  • variant (str, optional): Only MindSpore is supported. If the variant file name is specified, the weight is loaded from the variant file name.

  • download_checkpoint (bool, optional, defaults to True): Only MindSpore is supported. (Only origin_mode is supported.) Indicates whether to download the checkpoint file of a model.

Example:

from openmind import AutoConfig, AutoModelForSequenceClassification

# Download a configuration from openMind Models.
model  = AutoModelForSequenceClassification.from_pretrained("PyTorch-NPU/bert_base_cased")

# Update the configuration during loading.
model = AutoModelForSequenceClassification.from_pretrained("PyTorch-NPU/bert_base_cased", output_attentions=True)
model.config.output_attentions
'''
Output:
True
'''