AirflowProviderDeprecationWarning: Configuration 'namespace' is deprecated.

[kosteev]

Apache Airflow Provider(s)

cncf-kubernetes

Versions of Apache Airflow Providers

In one of the latest cncf-kubernetes provider we emit warning message:
"""
/opt/python3.11/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py:155 AirflowProviderDeprecationWarning: Configuration 'namespace' is deprecated. Use 'pod_template_file' to specify the namespace instead.
"""

airflow/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/kube_config.py

Line 85 in 875a427

"Configuration 'namespace' is deprecated. "

However it is still needed/used in kubernetex_executor_utils.py:
https://github.com/apache/airflow/blob/875a427ab3f1cc06bf470a9cebdd27aa8d9528a5/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/executors/kubernetes_executor_utils.py#L523C53-L523C67

So, this property is actually needed and shouldn't be deprecated?

Apache Airflow version

3.1.0

Operating System

Linux

Deployment

Google Cloud Composer

Deployment details

No response

What happened

No response

What you think should happen instead

No response

How to reproduce

Use latest cncf-kubernetes provider and run Airflow with KubernetesExecutor

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[romsharon98]

Hi @kosteev ,
This is the motivation for the change.
They are still there because we still don't want to make a breaking change in the package.
This is a warning for the user to know in the future versions those parameters will be removed.

[romsharon98]

close as deprecated
#59556

[kosteev]

Wait, I didn't get it.

The namespace parameter will be removed in favor of what?
I get that you can define namespace parameter in the spec of pod_template file, but this parameter also needed for KubernetesExecutor to watch for pod events (as I mentioned in the issue). So, how KubernetesExecutor will know from which namespace to listen for events?

Or let me rephrase the question. What should I do as a user when I see this deprecation warning message? If I stop providing namespace property, KubernetesExecutor then fails for me with permission error message, because it tries to use "default" namespace, not the custom namespace that I want it to use.

[potiuk]

100% with @kosteev -> I think this parameter was deprecated by mistake and deprecation should be reverted.

This is the places where it is used:

        if self.kube_config.multi_namespace_mode:
            if self.kube_config.multi_namespace_mode_namespace_list:
                namespaces = self.kube_config.multi_namespace_mode_namespace_list
            else:
                namespaces = [None]
        else:
            namespaces = [self.kube_config.kube_namespace]

and:

class AirflowKubernetesScheduler(LoggingMixin):
    """Airflow Scheduler for Kubernetes."""

    def __init__(
        self,
        kube_config: Any,
        result_queue: Queue[KubernetesResults],
        kube_client: client.CoreV1Api,
        scheduler_job_id: str,
    ):
        super().__init__()
        self.log.debug("Creating Kubernetes executor")
        self.kube_config = kube_config
        self.result_queue = result_queue
        self.namespace = self.kube_config.kube_namespace
        self.log.debug("Kubernetes using namespace %s", self.namespace)
        self.kube_client = kube_client
        self._manager = multiprocessing.Manager()
        self.watcher_queue = self._manager.Queue()
        self.scheduler_job_id = scheduler_job_id
        self.kube_watchers = self._make_kube_watchers()