(dZddlmZddlmZddlmZddlZddlZddlmZm Z m Z ddl m Z ddl Z ddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZdd lmZddlmZddlmZddlmZddlmZGddej:Zy)zThe BigQuery CLI copy command.)absolute_import)division)print_functionN)ListOptionalTuple)flags)bigquery_client)client_dataset) client_job) client_table)utils)bigquery_command)bq_cached_client) utils_flags)utils_formatting)bq_error) bq_id_utilsc feZdZdZdZdZdZdZdZdZ dZ d e d e jffd Zd ej d eej&j(deeee ffdZd ej d eej&j(de dej&j(de def dZde de deefdZxZS)Copyz5cp [-n] [,]* z**** NOTE! **** z8Dataset %s not found. Please enter a valid dataset name.z\Warning: This operation is a cross-region copy operation. This may incur additional charges.zWarning: This operation is a cross-region copy operation. This may incur additional charges and take a long time to complete. This command is running in sync mode. It is recommended to use async mode (-sync=false) for cross-region copy operation.z1cp: Proceed with cross-region copy of %s? [y/N]: z7cp: Table %s already exists. Replace the table? [y/N]: z %s, exiting.namefvctt| ||tjdddd|tjdddd|tjd dd d |tj d dd |tjdddd|tjdddd|tj ddd|tjddd||j|y)N no_clobberFz#Do not overwrite an existing table.n) short_name flag_valuesforcez1Ignore existing destination tables, don't prompt.f append_tablezAppend to an existing table.adestination_kms_keyz;Cloud KMS key for encryption of the destination table data.)rsnapshotz(Create a table snapshot of source table.srestorezNRestore table snapshot to a live table. Deprecated, please use clone instead.r expirationz?Expiration time, in seconds from now, of the destination table.clonezCreate a clone of source table.)superr__init__r DEFINE_boolean DEFINE_stringDEFINE_integer_ProcessCommandRc)selfrr __class__s $platform/bq/frontend/command_copy.pyr*z Copy.__init__6s $tR(  -   ;   &   E    2       I   9r 2clientsource_referencesreturnc d}d}t|D]y\}}|j}tj|j|}|0t j |jt|fzddig||}n||k7std}||fS||fS)aChecks whether all source datasets are from same region. Args: client: Bigquery client source_references: Source reference Returns: true - all source datasets are from the same region. Includes the scenario in which there is only one source dataset false - all source datasets are not from the same region. Raises: bq_error.BigqueryNotFoundError: If unable to compute the dataset region TN apiclient referencereasonnotFoundF) enumerateGetDatasetReferencer GetDatasetRegionr8rBigqueryNotFoundError_DATASET_NOT_FOUNDstr) r/r3r4"all_source_datasets_in_same_regionfirst_source_region_valsource_dataset source_regions r1:_CheckAllSourceDatasetsInSameRegionAndGetFirstSourceRegionz?Copy._CheckAllSourceDatasetsInSameRegionAndGetFirstSourceRegionms&*.&-.3..0n$55$$"m  ,,  # #s>':&< < z "    $+ - /-2* -/B BB#/" ./B BBr2source_references_strdest_referencedestination_regionc|j} |j||\}}|!tj|j|}|0t j|jt|fzddig|r||k(ryt|jtjjrd|jznd|j z|j"rydt%j&|j(|fzk7rt|j*|fzy y#t j $r!} tdt| zYd} ~ yd} ~ wwxYw) aChecks if it is a Cross Region Copy operation and obtains confirmation. Args: client: Bigquery client source_references: Source reference source_references_str: Source reference string dest_reference: Destination dataset reference destination_region: Destination dataset region Returns: true - it is not a cross-region operation, or user has used force option, or cross-region operation is verified confirmed with user, or Insufficient permissions to query datasets for validation false - if user did not allow cross-region operation, or Dataset does not exist hence operation can't be performed. Raises: bq_error.BigqueryNotFoundError: If unable to compute the dataset region Nr7z^Unable to determine source or destination dataset location, skipping cross-region validation: Tr:r; yF)r=rHr r>r8rBigqueryAccessDeniedErrorprintrAr?r@_NOTEbq_flagsSYNCHRONOUS_MODEvalue_SYNC_FLAG_ENABLED_WARNING_CROSS_REGION_WARNINGrfrontend_utilsPromptYN_CONFIRM_CROSS_REGION _NOT_COPYING) r/r3r4rIrJrKdestination_datasetrBrCerrs r1#shouldContinueAfterCrossRegionCheckz(Copy.shouldContinueAfterCrossRegionChecksn6)<<>  I I' >(*=  #+<<&&2E !  * *  ! !S)<%=$? ? Z   *11    $ $ * * t... D.. .   zz  n%% ""&;%== D  !6 8 89  =  - -  ' H   s8DE1E  E source_tables dest_tablec tjj}|jdDcgc]}t j ||}}dj d|D}t j ||}|jrd}d} n|jrd}d} nd}d } d } | sn|jsbtj|j| } | r?d tj|j|zk7rt!|j"|fzy |j%||||| sy d} |j&rd} d} n$|j(rd} d} n|j*rd} d} nd} || t-j.| d} t0j2j4rt0j2j4| d<|j6rd|j6i| d<|j8r_t:j:j=t?|j8tAj@z}tjB|| d<tEjF|||fi| }|t!d|dy t0jHj4s|jK|y tM|dkDrdnd}t!d |d!|d"| d#|d$ tjNtQjR|y cc}w)%zCopies one table to another. Examples: bq cp dataset.old_table dataset2.new_table bq cp --destination_kms_key=kms_key dataset.old_table dataset2.new_table ,) id_fallbacks identifierz, c32K|]}t|yw)N)rA).0srcs r1 z#Copy.RunWithArgs..s%L:K3c#h:Ks WRITE_APPENDT WRITE_EMPTYWRITE_TRUNCATEFNr7rNrcopiedSNAPSHOT snapshottedRESTORErestoredCLONEclonedCOPY)write_dispositionignore_already_existsjob_idoperation_typelocation kmsKeyNameencryption_configurationdestination_expiration_timezTable 'z' already exists, skippingr$Tablez 'z' successfully z to '')*rClientGetsplitbq_client_utilsGetTableReferencejoinr rrr get_table_regionr8rWrX_CONFIRM_OVERWRITErPrZr]r#r%r(rget_job_id_from_flagsrRLOCATIONrTr"r'datetimeutcfromtimestampinttime FormatRfc3339r CopyTablerSPrintJobStartInfolenPrintJobMessagesrformat_job_info)r/r^r_r3rfr4rIrJrsrtrK operationrvkwds datetime_utcjob pluralitys r1 RunWithArgszCopy.RunWithArgss $ $ ( ( *F!&&s++C ))v#N+!II%L:K%LL$66 N (" '"*# '88$$ ~'>'>  ! !^ 4(! d#8"::;  3 3  I }}!ni  ni nin.!6335(  D !**00d:  00*d %& &&77 doo + ,l-;,H,H -d ()   !> 59 C { ^ EF  & & , , S!./!3#i -y. J %%&6&F&Fs&KLmsK')__name__ __module__ __qualname__usagerQr@rVrUrYrrZrAr FlagValuesr*r BigqueryClientrrApiClientHelperTableReferencerboolrrHr]rr __classcell__)r0s@r1rr!s2 E% %@ * NP ,535E$4$45n&C,,&Ck99HHI&C T8C= ! &CPE,,Ek99HHIE! E "11@@ E  E EN^Ms^M^M ^Mr2r)__doc__ __future__rrrrrtypingrrrabslr rRclientsr r r r rrfrontendrrrWrrrr BigqueryCmdrr2r1rs^$&% ((#" ,%%, %YM  ' 'YMr2