&dZddlmZddlmZddlmZddlZddlZddlmZddl m Z ddl m Z ddl m Z dd l mZGd d e j Zy) zThe BigQuery CLI mkdef command.)absolute_import)division)print_functionN)Optional)flags)bigquery_command)utilscdeZdZdZdedej ffd Z d dedeedee fdZ xZ S) MakeExternalTableDefinitionzmkdef []namefvctt| ||tjddd|tjdddd|tj ddd |tj d dd |tjd dd |tj ddgdd|tj ddd|tjddd|tjddd|tjddtj|tj ddddgd|tj ddd d!gd"|tjd#dd$|tj d%dd&|tj d'dgd(d)d*|tj d+dd,d-gd.|tj||_ tj||_ tj||_tj ||_tj$||_tj(||_tj,||_tj0||_tj4||_|j9|y)/N autodetectz1Should schema and format options be autodetected.) flag_valuesignore_unknown_valuesz>Ignore any values in a row that are not present in the schema.i) short_namerhive_partitioning_modezEnables hive partitioning. AUTO indicates to perform automatic type inference. STRINGS indicates to treat all hive partition keys as STRING typed. No other values are accepted#hive_partitioning_source_uri_prefixzPrefix after which hive partition encoding begins. For URIs like gs://bucket/path/key1=value/file, the value should be gs://bucket/path.require_hive_partition_filterzbWhether queries against a table are required to include a hive partition key in a query predicate. source_formatCSV) r GOOGLE_SHEETSNEWLINE_DELIMITED_JSONDATASTORE_BACKUP DELTA_LAKEORCPARQUETAVROICEBERGzFormat of source data. Options include: CSV GOOGLE_SHEETS NEWLINE_DELIMITED_JSON DATASTORE_BACKUP DELTA_LAKE ORC PARQUET ICEBERG AVRO connection_idaThe connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection_id can have the form ".." or "projects//locations//connections/".use_avro_logical_typesTzIf sourceFormat is set to "AVRO", indicates whether to enable interpreting logical types into their corresponding types (ie. TIMESTAMP), instead of only using their raw types (ie. INTEGER).parquet_enum_as_stringFzHInfer Parquet ENUM logical type as STRING (instead of BYTES by default).parquet_enable_list_inferencemetadata_cache_mode AUTOMATICMANUALzEnables metadata cache for an external table with a connection. Specify AUTOMATIC to automatically refresh the cached metadata. Specify MANUAL to stop the automatic refresh.object_metadata DIRECTORYSIMPLEz/Object Metadata Type. Options include: SIMPLE.!preserve_ascii_control_characterszLWhether to preserve embedded Ascii Control characters in CSV External table reference_file_schema_urizqprovide a referencing file with the expected table schema, currently enabled for the formats: AVRO, PARQUET, ORC.encoding)zUTF-8z ISO-8859-1zUTF-16BEzUTF-16LEzUTF-32BEzUTF-32LEzThe character encoding used by the input file. Options include: ISO-8859-1 (also known as Latin-1) UTF-8 UTF-16BE (UTF-16 BigEndian) UTF-16LE (UTF-16 LittleEndian) UTF-32BE (UTF-32 BigEndian) UTF-32LE (UTF-16 LittleEndian)Efile_set_spec_typeFILE_SYSTEM_MATCHNEW_LINE_DELIMITED_MANIFESTaUSpecifies how to discover files given source URIs. Options include: FILE_SYSTEM_MATCH: expand source URIs by listing files from the underlying object store. This is the default behavior. NEW_LINE_DELIMITED_MANIFEST: indicate the source URIs provided are new line delimited manifest files, where each line contains a URI with no wild-card.)superr __init__rDEFINE_boolean DEFINE_string DEFINE_enumfrontend_utils"PARQUET_LIST_INFERENCE_DESCRIPTIONfrontend_flagsdefine_null_markernull_marker_flagdefine_null_markersnull_markers_flagdefine_time_zonetime_zone_flagdefine_date_formatdate_format_flagdefine_datetime_formatdatetime_format_flagdefine_time_formattime_format_flagdefine_timestamp_formattimestamp_format_flagdefine_source_column_matchsource_column_match_flagdefine_parquet_map_target_typeparquet_map_target_type_flag_ProcessCommandRc)selfr r __class__s %platform/bq/frontend/command_mkdef.pyr3z$MakeExternalTableDefinition.__init__s %t5dB?  ;    H     H  -  0  '  =     14         P     )  ' 99    h 8  h:  +    #  7   O ,   ;<  +=="MD+??BOD(99bID*=="MD . E E!D+=="MD!/!G!G"D%3$M$M%D! 55"E % 2 source_urisschemareturnctjtjd id|jd|d|d|j d|j d|jd|jd|jd |jd |jd |jd |jd |jd|jd|j d|j"d|j$d|j&d|j(j*d|j,j*d|j.j*d|j0j*d|j2j*d|j4j*d|j6j*d|j8j*d|j:j*t<j>ddy)!aEmits a definition in JSON for an external table, such as GCS. The output of this command can be redirected to a file and used for the external_table_definition flag with the "bq query" and "bq mk" commands. It produces a definition with the most commonly used values for options. You can modify the output to override option values. The argument is a comma-separated list of URIs indicating the data referenced by this external table. The argument should be either the name of a JSON file or a text schema. In the case that the schema is provided in text form, it should be a comma-separated list of entries of the form name[:type], where type will default to string if not specified. In the case that is a filename, it should be a JSON file containing a single array, each entry of which should be an object with properties 'name', 'type', and (optionally) 'mode'. For more detail: https://cloud.google.com/bigquery/docs/schemas#specifying_a_json_schema_file Note: the case of a single-entry schema with no type specified is ambiguous; one can use name:string to force interpretation as a text schema. Usage: mkdef [] Examples: bq mkdef 'gs://bucket/file.csv' field1:integer,field2:string Arguments: source_uris: Comma-separated list of URIs. schema: Either a text schema or JSON file, as above. rrQrRrr!rrrrr"r#r$r%r(r+r,r-r/ null_marker null_markers time_zone date_formatdatetime_format time_formattimestamp_formatsource_column_matchparquet_map_target_typeT) sort_keysindentN) jsondumpr7CreateExternalTableDefinitionrrr!rrrrr"r#r$r%r(r+r,r-r/r;valuer=r?rArCrErGrIrKsysstdout)rMrQrRs rO RunWithArgsz'MakeExternalTableDefinition.RunWithArgssP II44 ,, #    ,,  #'"<"<  $(#>#> 150X0X +/*L*L $(#>#> $(#>#> +/*L*L !% 8 8 !00 /3.T.T '+&D&D! "]]# $ $66% &--33' (//55) *))//+ ,--33- .!55;;/ 0--331 2"77==3 4!% = = C C5 6%)$E$E$K$K7 : A!rP)N) __name__ __module__ __qualname__usagestrr FlagValuesr3rintrh __classcell__)rNs@rOr r sS -%j3jE$4$4jZ7;II&.smI}IrPr )__doc__ __future__rrrrbrftypingrabslrfrontendrr9r r7 BigqueryCmdr rarPrOrws;%&% %,,x"2">">xrP