^.dZddlmZddlmZddlmZddlZddlZddlmZm Z m Z m Z ddl m Z ddlmZdd lmZdd lmZdd lmZdd lmZej*Zd ZdZGddZde j2defdZde j2defdZ d=de j2de ej:j<de ede ede ede ef dZ d>de j2de ej:jBde ede ede ede ef dZ" d>de j2dej:jFd e ede ede ef d!Z$de j2dej:jBd"e ed#e ed$e ef d%Z%de j2dd&d'e&fd(Z'de j2d)ed*efd+Z( d?de j2d,e d-d.e efgdej:jBd/e ed0e ed1e ed2e ed3e e eefd4e ed5e ed6e ed7e efd8Z) d@de j2ded9ed/e ed0e ed1e ed2e ed3e e eefd4e ed6e ed7e ed5e ede efd:Z* dAde j2dej:jBd;e&fd<Z+y)Bz.The BigQuery CLI data transfer client library.)absolute_import)division)print_functionN)AnyDict NamedTupleOptional) discovery)utils) bq_api_utils)bq_error) bq_id_utils)bq_processor_utilsauthorization_code version_infoc eZdZdZ ddeedeedeedeedeef dZ dd eeee fd eee ffd Z dd eeeefd eee ffd Z d efdZ dZ ded eeeffdZy)TransferScheduleArgsz.Arguments to customize data transfer schedule.Nschedule start_timeend_timedisable_auto_schedulingevent_driven_schedulecJ||_||_||_||_||_yN)rrrrr)selfrrrrrs +platform/bq/clients/client_data_transfer.py__init__zTransferScheduleArgs.__init__s)DM DODM#:D !6Doptions_to_copyreturnc|ji}|jr |j|j|d<|S|jri|d<|Si|d<|rd|vrt |d|d<|j r|j |dd<|j r!|j|j |dd<|jr!|j|j|dd<|S)aMReturns a dictionary of schedule options v2. Args: options_to_copy: Existing options to copy from. Returns: A dictionary of schedule options v2 expected by the bigquery.transfers.create and bigquery.transfers.update API methods. Raises: bq_error.BigqueryError: If shedule options conflict. eventDrivenSchedulemanualScheduletimeBasedScheduler startTimeendTime) _validate_schedule_optionsr_process_event_driven_schedulerdictrr_time_or_infitityrrroptionss rto_schedule_options_v2_payloadz3TransferScheduleArgs.to_schedule_options_v2_payload,s  ##%G !!'+'J'J  $ $(g#$* N%  % %"$g" N&(g!" 0OC'+ / 0( #$ 37==#$Z0 484J4J OO5 #$[1 262H2H MM3 #$Y/ Nrct|xsi}|j|j|j|d<|j|j|j|d<|j|d<|S)zReturns a dictionary of schedule options. Args: options_to_copy: Existing options to be copied. Returns: A dictionary of schedule options expected by the bigquery.transfers.create and bigquery.transfers.update API methods. r%r&disableAutoScheduling)r)rr*rrr+s rto_schedule_options_payloadz0TransferScheduleArgs.to_schedule_options_payloadVso?(b)G "!33DOODgk }} 11$--@gi'+'C'CG #$ Nrtime_strc|xsdS)zCReturns None to indicate Inifinity, if time_str is an empty string.N)rr1s rr*z&TransferScheduleArgs._time_or_infitityos  trct|j|j|jg}|jdu}t |j ||gdkDrtjdy)zsValidates schedule options. Raises: bq_error.BigqueryError: If the given schedule options conflict. NzThe provided scheduling options conflict. Please specify one of no_auto_scheduling, time-based schedule or event-driven schedule.) anyrrrrsumrr BigqueryError)ris_time_based_scheduleis_event_driven_schedules rr'z/TransferScheduleArgs._validate_schedule_optionsss~ ! 7 $99E  ( ( " $     " " O  rc tj|}d|vrtjd|S#t$r}tjd|d}~wwxYw)aProcesses the event_driven_schedule given in JSON format. Args: event_driven_schedule: The user specified event driven schedule. This should be in JSON format given as a string. Ex: --event_driven_schedule='{"pubsub_subscription":"subscription"}'. Returns: parsed_event_driven_schedule: The parsed event driven schedule. Raises: bq_error.BigqueryError: If there is an error with the given params. z9Event driven schedule should be specified in JSON format.Npubsub_subscriptionz 4      798<<(:B?@% X %#m*<{*J&\\/" 8<<8 99  rrrrun_timecPtj|tjjdt |}|rd|i}nd||di}|j j j}|j||j}|jdS)adStarts manual transfer runs. Args: transfer_client: the transfer client to use. reference: Transfer configuration name for the run. start_time: Start time of the range of transfer runs. end_time: End time of the range of transfer runs. run_time: Specific time for a transfer run. Returns: The list of started transfer runs. start_manual_transfer_runsr_requestedRunTimerequestedTimeRange)r%r&)parentbodyrV) rrbrcrvrFrNrOrPstartManualRunsrRrQ) rHrXrrrrrconfigs_requestrxs rrr~s&!!99 ) y>&  )D J8L D$,,.88:JJL/  , ,F , F N N P( f rz3bq_id_utils.ApiClientHelper.TransferConfigReferencer c@tj|tjjd |j j j j|jjy#tj$rYywxYw)z$Returns true if the transfer exists.transfer_existsr_rLTF) rrbrcrvrNrOrPrQtransferConfigNamerRr BigqueryNotFoundError)rHrXs rrrs !!99  ((*::<@@  ) )A gi   ' ' sABBBproject_referencedata_source_idc|dz|z}|jjjj|j S)Nz/locations/-/dataSources/rL)rNrO dataSourcesrQrR)rHrrdata_source_retrievals r_fetch_data_sourcersL 55F  y{{} s%s&wy r id_fallbacksIDS project_idtarget_dataset display_namerefresh_window_daysparams auth_infoservice_account_namedestination_kms_keynotification_pubsub_topic schedule_argsc tj|tjjddt j |j z} t||j} g}i}| d|d<|r||d<|jd|r|jd||d <|r'tj||}|jd |r,J&BB~& L AB44   2 3 $% :;0IL,-BC1Y>)*|y8|$-."560L+,>?((*::<BB   ) )88K(%D9==9K+L%-D9==3N/C gi  ( (.#a&8)==#  * *. 08Z2H"   9~s BI!!K4AKK data_sourcec ri} |r|| d<|r|| d<ntjd|| d<|r%t|||}tj||| |} |rtj || } ntjd| r |dz| z}n|dz}| r| j | d<| r| | d <| r d d | ii| d <|jjjj|| |d n|jt|d n|jt|j}|dS)aCreate a transfer config corresponding to TransferConfigReference. Args: transfer_client: the transfer client to use. reference: the TransferConfigReference to create. data_source: The data source for the transfer config. target_dataset: The dataset where the new transfer config will exist. display_name: A display name for the transfer config. refresh_window_days: Refresh window days for the transfer config. params: Parameters for the created transfer config. The parameters should be in JSON format given as a string. Ex: --params="{'param':'value'}". The params should be the required values needed for each data source and will vary. auth_info: A dict contains authorization info which can be either an authorization_code or a version_info that the user input if they need credentials. service_account_name: The service account that the user could act as and used as the credential to create transfer runs from the transfer config. notification_pubsub_topic: The Pub/Sub topic where notifications will be sent after transfer runs associated with this transfer config finish. schedule_args: Optional parameters to customize data transfer schedule. destination_kms_key: Optional KMS key for encryption. location: The location where the new transfer config will run. Raises: BigqueryNotFoundError: if a requested item is not found. bq_error.BigqueryError: if a required field isn't provided. Returns: The generated transfer configuration name. rrz A display name must be provided.rzParameters must be provided.z /locations/z /locations/-rrrrrN)rrrrrrM)r r8rrrrr-rNrOrPcreaterQrrrR)rHrXrrrrrrrrrrrY create_itemsrrnew_transfer_configs rcreate_transfer_configrPs\,+9L'(".L  !C DD!,,~)K&BB-|[L  %77 ML  !? @@  & 1F 'F446$%0IL,-"560L+,  y{ v'dY]];M-N }}\*1 wy$ V $$rignore_not_foundctj|tjjd |j j j j|jjy#tj$r)}|stjd|ddig|Yd}~yd}~wwxYw)aDeletes TransferConfigReference reference. Args: transfer_client: the transfer client to use. reference: the TransferConfigReference to delete. ignore_not_found: Whether to ignore "not found" errors. Raises: BigqueryTypeError: if reference is not a TransferConfigReference. bq_error.BigqueryNotFoundError: if reference does not exist and ignore_not_found is False. delete_transfer_configr_rLrrrN) rrbrcrvrNrOrPdeleterrRr r)rHrXrrAs rrrs$!!99 % ((*::<CC  ) )D gi  ' '   * *& (8Z*@"   sABCB>>C)NNNNN)NNN) NNNNNNNNN) NNNNNNNNNN)F),rE __future__rrrr=rktypingrrrr googleapiclientr clientsr rr r rrServicerrrResourcerFrSrWrcrdintr^rvrtTransferRunReferencerrrGrrrrrr3rrrs4&% 22%,$   * I(I(X););#i&8&8cIM"# $%) D''D 33DDEDsmD} D  D c] DV"& $ ;''; 33KKL;#;# ;  ; SM ;B#'!% $ :'':**??:3-:# :  :z$''$**BB$ $sm $ sm $N''D (''6%)"&)- *.*.)-/348#B''B 8C= ) B**BBBSMB3-B"#B SMBS#X'B#3-B"#B (}!B"01#BR%)"&)- *.*./348)-"q%''q%q%q%SM q% 3- q% "# q% SMq%S#X'q%#3-q% (}q%01q%"#q%smq%n#''**BBr