?NdZddlmZddlmZddlmZddlmZddlmZ ddlm Z ddl m Z ddlm Zdd lmZdd lZd Ze j(j*fd Zd d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d d e j(j*d d d d d d d d d d d d d d d d d d d d d d d f?d Ze j(j*fdZe j(j*fdZe j(j*fdZe j(j*fdZe j(j*fdZe j(j*fdZe j(j*fdZe j(j*fdZe j(j*fdZ e j(j*fdZ!e j(j*fdZ"e j(j*fdZ#dZ$dZ%dZ&dZ'dZ(e j(j*fdZ)e j(j*fd Z*d!Z+d"Z,d#Z-y )$zACommon utility functions for Composer environment patch commands.)absolute_import)division)unicode_literals)environments_util)operations_util)util)base)logNctj|}|j}|dk(rF|j|j |jj j |_nF|j|j ||jj j|_d|j|fS)aConstructs an environment patch for Airflow Database Retention feature. Args: airflow_database_retention_days: int or None, the number of retention days for airflow database data retention mechanism release_track: base.ReleaseTrack, the release track of command. It dictates which Composer client library is used. Returns: (str, Environment), the field mask and environment to use for update.  release_trackr) retentionMode)airflowMetadataRetentionConfig) retentionDaysrz>config.data_retention_config.airflow_metadata_retention_configconfig) api_utilGetMessagesModuleEnvironmentConfigDataRetentionConfig$AirflowMetadataRetentionPolicyConfigRetentionModeValueValuesEnumRETENTION_MODE_DISABLEDdataRetentionConfigRETENTION_MODE_ENABLED Environment)airflow_database_retention_daysr messagesrs Alib/googlecloudsdk/command_lib/composer/environment_patch_util.py+_ConstructAirflowDatabaseRetentionDaysPatchr s ' 'm D(  % % '&$)!)!=!='/'T'T"GGdd||(U( ">"F "*!=!='/'T'T9"GGdd{{(U( ">"FG&) c tj||||}dj|j}|r)t j |j dd||S tj|dj|j |j|y#tj$rK}tjdj|j tj|d}~wwxYw) a Patches an Environment, optionally waiting for the operation to complete. This function is intended to perform the common work of an Environment patching command's Run method. That is, calling the patch API method and waiting for the result or immediately returning the Operation. Args: env_resource: googlecloudsdk.core.resources.Resource, Resource representing the Environment to be patched field_mask: str, a field mask string containing comma-separated paths to be patched patch: Environment, a patch Environment containing updated values to apply is_async: bool, whether or not to perform the patch asynchronously release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: an Operation corresponding to the Patch call if `is_async` is True; otherwise None is returned after the operation is complete Raises: command_util.Error: if `is_async` is False and the operation encounters an error r zwith operation [{0}] environmentT)kindis_asyncdetailsz(Waiting for [{}] to be updated with [{}]zError updating [{}]: {}N) environments_api_utilPatchformatnamer UpdatedResource RelativeNameoperations_api_utilWaitForOperation command_utilErrorsix text_type) env_resource field_maskpatchr%r operationr&es rr(r(As:$))E:]D) " ) )).. 9' !!#   8((299  % % ' 9# %   8   6==!!#S]]1%57 888s!AB""D5AC;;Dc@|r t||(S|$r t|$|(S|r"tdgtj||(S|s|s|rt||||(S| t1|>|(S|? t3|?|(Stj(d)aConstructs an environment patch. Args: is_composer_v1: boolean representing if patch request is for Composer 1.*.* Environment. env_ref: resource argument, Environment resource argument for environment being updated. node_count: int, the desired node count update_pypi_packages_from_file: str, path to local requirements file containing desired pypi dependencies. clear_pypi_packages: bool, whether to uninstall all PyPI packages. remove_pypi_packages: iterable(string), Iterable of PyPI packages to uninstall. update_pypi_packages: {string: string}, dict mapping PyPI package name to extras and version specifier. clear_labels: bool, whether to clear the labels dictionary. remove_labels: iterable(string), Iterable of label names to remove. update_labels: {string: string}, dict of label names and values to set. clear_airflow_configs: bool, whether to clear the Airflow configs dictionary. remove_airflow_configs: iterable(string), Iterable of Airflow config property names to remove. update_airflow_configs: {string: string}, dict of Airflow config property names and values to set. clear_env_variables: bool, whether to clear the environment variables dictionary. remove_env_variables: iterable(string), Iterable of environment variables to remove. update_env_variables: {string: string}, dict of environment variable names and values to set. update_image_version: string, image version to use for environment upgrade update_web_server_access_control: [{string: string}], Webserver access control to set cloud_sql_machine_type: str or None, Cloud SQL machine type used by the Airflow database. web_server_machine_type: str or None, machine type used by the Airflow web server scheduler_cpu: float or None, CPU allocated to Airflow scheduler. Can be specified only in Composer 2.0.0. worker_cpu: float or None, CPU allocated to each Airflow worker. Can be specified only in Composer 2.0.0. web_server_cpu: float or None, CPU allocated to Airflow web server. Can be specified only in Composer 2.0.0. scheduler_memory_gb: float or None, memory allocated to Airflow scheduler. Can be specified only in Composer 2.0.0. worker_memory_gb: float or None, memory allocated to each Airflow worker. Can be specified only in Composer 2.0.0. web_server_memory_gb: float or None, memory allocated to Airflow web server. Can be specified only in Composer 2.0.0. scheduler_storage_gb: float or None, storage allocated to Airflow scheduler. Can be specified only in Composer 2.0.0. worker_storage_gb: float or None, storage allocated to each Airflow worker. Can be specified only in Composer 2.0.0. web_server_storage_gb: float or None, storage allocated to Airflow web server. Can be specified only in Composer 2.0.0. min_workers: int or None, minimum number of workers in the Environment. Can be specified only in Composer 2.0.0. max_workers: int or None, maximumn number of workers in the Environment. Can be specified only in Composer 2.0.0. scheduler_count: int or None, number of schedulers in the Environment. Can be specified only in Composer 2.0.0. clear_maintenance_window: bool or None, specifies if maintenance window options should be cleared. maintenance_window_start: Datetime or None, a starting date of the maintenance window. maintenance_window_end: Datetime or None, an ending date of the maintenance window. maintenance_window_recurrence: str or None, recurrence RRULE for the maintenance window. environment_size: str or None, one of small, medium and large. master_authorized_networks_enabled: bool or None, whether the feature should be enabled master_authorized_networks: iterable(string) or None, iterable of master authorized networks. airflow_database_retention_days: Optional[int], the number of retention days for airflow database data retention mechanism. Infinite retention will be applied in case `0` or no integer is provided. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. triggerer_cpu: float or None, CPU allocated to Airflow triggerer. Can be specified only in Airflow 2.2.x and greater. triggerer_memory_gb: float or None, memory allocated to Airflow triggerer. Can be specified only in Airflow 2.2.x and greater. triggerer_count: int or None, number of triggerers in the Environment. Can be specified only in Airflow 2.2.x and greater enable_scheduled_snapshot_creation: bool, whether the automatic snapshot creation should be enabled snapshot_location: str, a Cloud Storage location used to store automatically created snapshots snapshot_schedule_timezone: str, time zone that sets the context to interpret snapshot_creation_schedule. snapshot_creation_schedule: str, cron expression that specifies when snapshots will be created cloud_data_lineage_integration_enabled: bool or None, whether the feature should be enabled support_web_server_plugins: bool or None, whether to enable/disable the support for web server plugins support_private_builds_only: bool or None, whether to enable/disable the support for private only builds dag_processor_cpu: float or None, CPU allocated to Airflow dag processor. Can be specified only in Composer 3. dag_processor_count: int or None, number of Airflow dag processors. Can be specified only in Composer 3. dag_processor_memory_gb: float or None, memory allocated to Airflow dag processor. Can be specified only in Composer 3. dag_processor_storage_gb: float or None, storage allocated to Airflow dag processor. Can be specified only in Composer 3. disable_vpc_connectivity: bool or None, defines whether to disable connectivity with a user's VPC network. Can be specified only in Composer 3. network: str or None, the Compute Engine network to which to connect the environment specified as relative resource name. Can be specified only in Composer 3. subnetwork: str or None, the Compute Engine subnetwork to which to connect the environment specified as relative resource name. Can be specified only in Composer 3. network_attachment: str or None, the Compute Engine network attachment that is used as PSC Network entry point. workload_updated: bool or None, verify if workload config has been updated enable_private_environment: bool or None, defines whether the internet access is disabled from Composer components. Can be specified only in Composer 3. disable_private_environment: bool or None, defines whether the internet access is enabled from Composer components. Can be specified only in Composer 3. enable_high_resilience: bool or None, defines whether high resilience should be enabled for given environment. Can be specified only in Composer 2. enable_logs_in_cloud_logging_only: bool or None, defines whether logs in cloud logging only feature should be enabled for given environment. Can be specified only in composer 2. Returns: (str, Environment), the field mask and environment to use for update. Raises: command_util.Error: if no update type is specified r T)scheduler_countr ziYou cannot use Workloads Config flags introduced in Composer 2.X when updating Composer 1.X environments. scheduler_cpu worker_cpuweb_server_cpuscheduler_memory_gbworker_memory_gbweb_server_memory_gbscheduler_storage_gbworker_storage_gbweb_server_storage_gbworker_min_countworker_max_countr9r triggerer_cputriggerer_memory_gbtriggerer_countdag_processor_cpudag_processor_memory_gbdag_processor_countdag_processor_storage_gbz8Cannot update Environment with no update type specified.)_ConstructNodeCountPatch_ConstructEnvironmentSizePatch_ConstructPyPiPackagesPatchr/ParseRequirementsFile!_ConstructPrivateEnvironmentPatch_ConstructLabelsPatch_ConstructAirflowConfigsPatch_ConstructEnvVariablesPatch_ConstructImageVersionPatch%_ConstructWebServerAccessControlPatch"_ConstructCloudSqlMachineTypePatch#_ConstructWebServerMachineTypePatch+_ConstructMasterAuthorizedNetworksTypePatch _ConstructScheduledSnapshotPatch _ConstructPrivateBuildsOnlyPatch#_ConstructWebServerPluginsModePatch_ConstructVpcConnectivityPatchr 2_ConstructSoftwareConfigurationSchedulerCountPatchr0_ConstructAutoscalingPatch _ConstructMaintenanceWindowPatch?_ConstructSoftwareConfigurationCloudDataLineageIntegrationPatch_ConstructHighResiliencePatch%_ConstructLogsInCloudLoggingOnlyPatch)@is_composer_v1env_ref node_countupdate_pypi_packages_from_fileclear_pypi_packagesremove_pypi_packagesupdate_pypi_packages clear_labels remove_labels update_labelsclear_airflow_configsremove_airflow_configsupdate_airflow_configsclear_env_variablesremove_env_variablesupdate_env_variablesupdate_image_version update_web_server_access_controlcloud_sql_machine_typeweb_server_machine_typer:r;r<r=r>r?r@rArB min_workers max_workersr9clear_maintenance_windowmaintenance_window_startmaintenance_window_endmaintenance_window_recurrenceenvironment_size"master_authorized_networks_enabledmaster_authorized_networksrr rErFrG"enable_scheduled_snapshot_creationsnapshot_locationsnapshot_schedule_timezonesnapshot_creation_schedule&cloud_data_lineage_integration_enabledsupport_web_server_pluginssupport_private_builds_onlyrHrJrIrKdisable_vpc_connectivitynetwork subnetworknetwork_attachmentworkload_updatedenable_private_environmentdisable_private_environmentenable_high_resilience!enable_logs_in_cloud_logging_onlys@ rConstructPatchrtsV #Jm LL )  77# & b**+IJ# %%04H &#  %%  #> ,"# %%]m m]- QQ5 (#  %% 04H &#  %%  &M ;;%1 0(  GG -m == .} >>'3 6*,F (3 +,N,F,=,F,9  ;; !, +#]  + ."M *    )   %0 6' 88 ='} FF    6 77(%(2  ,  4  4.!6''*&&2 *!".#$#:%&2'($<). ' ! + % #  ,7 J.  ' (  '2 0)=  @ r!ctj|}|j}|j|}d}t ||_||j |fS)aConstructs an environment patch for private environment. Args: enable_private_environment: bool or None, defines whether the internet access is disabled from Composer components. Can be specified only in Composer 3. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r privateEnvironmentConfigzK):LK!K'K&) r!cztj|}|j|}d|j|fS)a@Constructs an environment patch for node count. Args: node_count: int, the desired node count release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r ) nodeCountzconfig.node_countrrrrr)rfr rrs rrMrM>s@ ' 'm D(  % % % ;& h22&2A AAr!cztj|}|j|}d|j|fS)aSConstructs an environment patch for environment size. Args: environment_size: str, the desired environment size. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r )environmentSizezconfig.environment_sizerr)r~r rrs rrNrNNsA ' 'm D(  % %6F % G& "H$8$8$8$G GGr!ctj|jjjj }fd}t j|||d||S)aOConstructs an environment patch for partially updating PyPI packages. Args: clear_pypi_packages: bool, whether to clear the PyPI packages dictionary. remove_pypi_packages: iterable(string), Iterable of PyPI package names to remove. update_pypi_packages: {string: string}, dict mapping PyPI package name to optional extras and version specifier. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r clj|}j|}|S)NadditionalProperties) pypiPackagessoftwareConfigrSoftwareConfigr)entriessoftware_configrenv_clsrpypi_packages_clss r _BuildEnvz._ConstructPyPiPackagesPatch.._BuildEnvv?--&GD.FO  ' ' ' GF & !!r!z$config.software_config.pypi_packages)rrrrPypiPackagesValueAdditionalPropertyr/BuildPartialUpdate) rhrirjr entry_clsrrrrs @@@rrOrO_si$ ' 'm D(  '..@@22)"  ( (/1E,i DDr!ctj|}|jjj}fd}t j |||d||S)aConstructs an environment patch for updating labels. Args: clear_labels: bool, whether to clear the labels dictionary. remove_labels: iterable(string), Iterable of label names to remove. update_labels: {string: string}, dict of label names and values to set. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r c6j|S)Nr)labels) LabelsValue)rrs rrz(_ConstructLabelsPatch.._BuildEnvs '--7-K LLr!r)rrrrrr/r)rkrlrmr rrrrs @rrRrRsZ  ' 'm D(  '!!44)M  ( (})6))2 44r!ctj|jjjj }fd}t j|||d||S)aPConstructs an environment patch for updating Airflow configs. Args: clear_airflow_configs: bool, whether to clear the Airflow configs dictionary. remove_airflow_configs: iterable(string), Iterable of Airflow config property names to remove. update_airflow_configs: {string: string}, dict of Airflow config property names and values to set. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r clj|}j|}|S)Nr)airflowConfigOverridesrrr)rrrairflow_config_overrides_clsrrs rrz0_ConstructAirflowConfigsPatch.._BuildEnvsC--;!( *.+O ' ' ' GF & !!r!z/config.software_config.airflow_config_overrides)rrrrAirflowConfigOverridesValuerr/r) rnrorpr rrrrrs @@@rrSrSsk& ' 'm D(  '99*==)"  ( (35K7I OOr!c d tj||}|jjj}|r |j ng}t j| j jj j} fd} dtj|||||| fS)aConstructs an environment patch for updating environment variables. Note that environment variable updates do not support partial update masks unlike other map updates due to comments in (b/78298321). For this reason, we need to retrieve the Environment, apply an update on EnvVariable dictionary, and patch the entire dictionary. The potential race condition here (environment variables being updated between when we retrieve them and when we send patch request)is not a concern since environment variable updates take 5 mins to complete, and environments cannot be updated while already in the updating state. Args: env_ref: resource argument, Environment resource argument for environment being updated. clear_env_variables: bool, whether to clear the environment variables dictionary. remove_env_variables: iterable(string), Iterable of environment variable names to remove. update_env_variables: {string: string}, dict of environment variable names and values to set. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r clj|}j|}|S)Nr) envVariablesrrr)rrrrenv_variables_clsrs rrz._ConstructEnvVariablesPatch.._BuildEnvrr!z$config.software_config.env_variables)r'GetrrrrrrrrEnvVariablesValuerr/BuildFullMapUpdate) rerqrrrsr env_objinitial_env_var_valueinitial_env_var_listrrrrrs @@@rrTrTs> " % %g] K'!..77DD 00$& ' 'm D(  '--??22)" 1  ) )*=*>*>*> *3  5 66r!c tj|}|j|j|j ||||}d|j |fS)aConstructs an environment patch for environment image version. Args: enable_scheduled_snapshot_creation: bool, whether the automatic snapshot creation should be enabled snapshot_creation_schedule: str, cron expression that specifies when snapshots will be created snapshot_location: str, a Cloud Storage location used to store automatically created snapshots snapshot_schedule_timezone: str, time zone that sets the context to interpret snapshot_creation_schedule. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r )enabledsnapshotCreationSchedulesnapshotLocationtimeZone)scheduledSnapshotsConfig)recoveryConfigz1config.recovery_config.scheduled_snapshots_configr)rrrRecoveryConfigScheduledSnapshotsConfigr)rrrrr rrs rrZrZs, ' 'm D(  % %,,#+#D#D8'A01 $E$3-4 & 5& =h>R>R ?S? r!c4tj|}|j}|r&|jjj|_n%|jjj |_|j|}d|j|fS)atConstructs an environment patch for web server plugins mode patch. Args: support_web_server_plugins: bool, defines if plugins are enabled or not. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r rz.config.software_config.web_server_plugins_moder) rrr#WebServerPluginsModeValueValuesEnumPLUGINS_ENABLEDwebServerPluginsModePLUGINS_DISABLEDrr)rr rrrs rr\r\s ' 'm D(++-/CCSS( CCTT(  % %_ % E& 98;O;O r?r@rArBrCrDr9r rErFrGrHrIrJrKrworkload_resourcesrs rr_r_s(p ' 'm D(**&( +! **')++ $ $#%## %%& !&.&@&@ $7'A'{#$)1)F)F (*! *G*~&  % %.h..D1CD & F& "H$8$8$8$G GGr!ctj|}|rd|jfS|j|j |j |}|j |}d|j|fS)aConstructs an environment patch for updating maintenance window. Args: maintenance_window_start: Datetime or None, a starting date of the maintenance window. maintenance_window_end: Datetime or None, an ending date of the maintenance window. maintenance_window_recurrence: str or None, recurrence RRULE for the maintenance window. clear_maintenance_window: bool or None, specifies if maintenance window options should be cleared. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r zconfig.maintenance_window) startTimeendTime recurrence)maintenanceWindowr)rrrMaintenanceWindow isoformatr)r{r|r}rzr r window_valuers rr`r`s, ' 'm D( &(<(<(> >>++(224$..0.,0,  % % % E& $h&:&:&&:&I IIr!ctj|}d|j|j|j |fS)aSConstructs a patch for updating scheduler count for Composer 1.*.*. Args: scheduler_count: number of schedulers. release_track: base.ReleaseTrack, the release track of command. Will dictate which Composer client library will be used. Returns: (str, Environment), the field mask and environment to use for update. r z&config.software_config.scheduler_count)schedulerCountrr)rrrrr)r9r rs rr^r^'s] ' 'm D( 183G3G  ' '!00,1.(/4H40 00r!c tj|}d|j|j|j |j |fS)arConstructs a patch for updating Cloud Data Lineage integration config. Args: enabled: bool, whether Cloud Data Lineage integration should be enabled. release_track: base.ReleaseTrack, the release track of command. It dictates which Composer client library is used. Returns: (str, Environment), the field mask and environment to use for update. r z5config.software_config.cloud_data_lineage_integration)r)cloudDataLineageIntegrationrr)rrrrrCloudDataLineageIntegrationrr rs rrara;ss ' 'm D( @(BVBV  ' '!00*2*N*N!+O+#1$(%CWC& &&r!ctj|}|s"d|j|jfSd|j|j|jjj fS)aMConstructs a patch for updating high resilience. Args: enabled: bool, whether High resilience should be enabled. release_track: base.ReleaseTrack, the release track of command. It dictates which Composer client library is used. Returns: (str, Environment), the field mask and environment to use for update. r zconfig.resilience_moder)resilienceMode)rrrrResilienceModeValueValuesEnumHIGH_RESILIENCErs rrbrbPs ' 'm D(  #X%9%9))+&:&  "8#7#7  ' '((FFVV($8$ r!cdtj|}|r1|j|jjj}n0|j|jjj }|j |}|j|}d|j|fS)ajConstructs a patch for updating logs in cloud logging only feature. Args: enabled: bool, whether logs in cloud logging onlyshould be enabled. release_track: base.ReleaseTrack, the release track of command. It dictates which Composer client library is used. Returns: (str, Environment), the field mask and environment to use for update. r ) storageMode)taskLogsRetentionConfig)rzDconfig.data_retention_config.task_logs_retention_config.storage_moder) rrTaskLogsRetentionConfigStorageModeValueValuesEnumCLOUD_LOGGING_ONLYCLOUD_LOGGING_AND_CLOUD_STORAGErrr)rr rtask_logs_retention_configdata_retention_configrs rrcrcjs ' 'm D( !)!A!A44OObb"B""*!A!A44OOoo"B"#6687  % %:O % P&L&) r!).__doc__ __future__rrrgooglecloudsdk.api_lib.composerrr'rr-rrgooglecloudsdk.callioper #googlecloudsdk.command_lib.composerr/googlecloudsdk.corer r1r ReleaseTrackGAr(rrQr[r]rMrNrOrRrSrTrZr\rUrVrWrXrYr_r`r^rarbrcrLr!rr'sH&'VR<(D#  N)),, 08j #'%) !!"&'+#$(##&&'+##+/# $ !! # $&*AZ~ ##&&D##&&J##&& *Z8<7H7H7K7K B"261B1B1E1EH(/3.?.?.B.BDJ)-(9(9(<(<4>150A0A0D0D"OR/3.?.?.B.B 56x483D3D3G3G H/3.?.?.B.B@/3.?.?.B.B(*((2ZHB483D3D3G3G !JJ$(#4#4#7#70(&*4r!