All Airflow Configurations set via Environment Variable are masked when expose_config is set as non-sensitive-only #28756

Activity

1. wolfier

   added

   area:core

   kind:bugThis is a clearly a bugThis is a clearly a bug

   on Jan 5, 2023
2. wolfier

   changed the title [-]All Airflow Configurations set via Environment Variable are masked when `expose_config` is set as `non-sensitive-only``[/-] [+]All Airflow Configurations set via Environment Variable are masked when `expose_config` is set as `non-sensitive-only`[/+] on Jan 5, 2023
3. wolfier commented on Jan 5, 2023

   

   wolfier

   on Jan 5, 2023

   ContributorAuthor

   More actions

   I believe the issue is because when the configuration is fetched as a dictionary, the values have already been hidden because display_sensitive is passed as False. The dictionary is built by calling _include_envs which hides any configuration that does not end with _cmd and _secret.

   Instead, conf.as_dict should be passed with display_sensitive as True. That way, all configurations are not hidden and can be selectively hidden with SENSITIVE_CONFIG_VALUES.

   conf_dict = conf.as_dict(True, True)
   
   for sect, key in SENSITIVE_CONFIG_VALUES:
       if sect in conf_dict and key in conf_dict[sect]:
           value, source = conf_dict[sect][key]
           conf_dict[sect][key] = ("< hidden >", source)
   
   table = [
       (section, key, str(value), source)
       for section, parameters in conf_dict.items()
       for key, (value, source) in parameters.items()
   ]
4. potiuk

   added

   good first issue

   on Jan 6, 2023
5. potiuk commented on Jan 6, 2023

   

   potiuk

   on Jan 6, 2023

   Member

   More actions

   Would you like to take a stab on it @wolfier ?
6. uranusjr

   added

   area:webserverWebserver related IssuesWebserver related Issues

   and removed

   area:core

   on Jan 6, 2023
7. ephraimbuddy commented on Jan 6, 2023

   

   ephraimbuddy

   on Jan 6, 2023

   Contributor

   More actions

   I think we should hide this in _include_envs so it's same as _include_cmds & _include_secrets:

   diff --git a/airflow/configuration.py b/airflow/configuration.py
   index 41778fe374..df393da6e6 100644
   --- a/airflow/configuration.py
   +++ b/airflow/configuration.py
   @@ -1149,7 +1149,8 @@ class AirflowConfigParser(ConfigParser):
                if not display_sensitive and env_var != self._env_var_name("core", "unit_test_mode"):
                    # Don't hide cmd/secret values here
                    if not env_var.lower().endswith("cmd") and not env_var.lower().endswith("secret"):
   -                    opt = "< hidden >"
   +                    if (section, key) in self.sensitive_config_values:
   +                        opt = "< hidden >"
    
                elif raw:
                    opt = opt.replace("%", "%%")
   (END)

   👍React with 👍1Reacted by Jarek Potiuk
8. uranusjr commented on Jan 6, 2023

   

   uranusjr

   on Jan 6, 2023

   Member

   More actions

   I searched for as_dict usages and this is the only place where display_sensitive is False, so I think we should simply incorporate SENSITIVE_CONFIG_VALUES as Ephraim said. Otherwise the flag is basically useless.

   👍React with 👍1Reacted by Jarek Potiuk
9. ephraimbuddy

   closed this as completedin #28802on Jan 9, 2023