dZddlmZddlmZddlmZddlZddlZddlZddlm Z ddl m Z ddl m Zdd lmZGd d Zd Zy) zBProvides split file preprocessing for adding splits to a database.)absolute_import)division)unicode_literalsN) extra_types)apis) exceptions)filesc.eZdZdZdZdZdZdZdZy)SplitFileParseraaParses a split file into a list of split points. The split file is expected to be in the format of: [space][space]() [space][space]() ... where ObjectType can be TABLE or INDEX. Each split point must be in a new line. Split value is expected to be a comma separated list of key parts. Split values should be surrounded by parenthesis like () String values should be supplied in single quotes:'splitKeyPart' Boolean values should be one of: true/false INT64 and NUMERIC spanner datatype values should be supplied within single quotes values like string format: '123', '999999999999999999999999999.99' Other number values should be supplied without quotes: 1.287 Timestamp values should be provided in the following format in single quote values: '2020-06-18T17:24:53Z' If the split value needs to have a comma, then that should be escaped by backslash. Examples: TABLE Singers ('c32ca57a-786c-2268-09d4-95182a9930be') INDEX Order (4.2) TABLE TableD (0,'7ef9db22-d0e5-6041-8937-4bc6a7ef9db2') INDEX IndexXYZ ('8762203435012030000',NULL,NULL) INDEX IndexABC (0, '2020-06-18T17:24:53Z') TableKey (123,'ab\,c') -- note that the above split value has a delimieter (comma) in it, hence escaped by a backslash. c$||_||_tjd|_tjd|_tjd|_tjd|_tjd|_y)Nz(\S+)\s+(\S+)\s+(.+)z"\((.*?)\) TABLE (\S+)\s+\((.*?)\)$z"\((.*?)\) INDEX (\S+)\s+\((.*?)\)$z\((.*?)\) TableKey \((.*?)\)$z \((.*?)\)$) splits_filesplit_expiration_daterecompilesplit_line_pattern&incorrect_split_with_table_key_pattern&incorrect_split_with_index_key_patternindex_full_key_patternsingle_key_pattern)selfr rs ;lib/googlecloudsdk/command_lib/spanner/split_file_parser.py__init__zSplitFileParser.__init__?ss"D!6D jj)@AD24**-3D/35**-3D/#%**-M"ND jj7Dctjdd}g}tj|j5}|j j D]}|j|}|r$|dr|dr|dr|djdvr%tjddj||j}|djd k(r |d|_ n |djd k(r |d|_|dr|j|d|_|j"r|j"|_|j'| d d d |S#1swY|SxYw) *Gets the split points from the input file.spannerv1 SplitValue ObjectName ObjectType)TABLEINDEX --splits-fileInvalid split point string: {}. Each split point must be in the format of () where ObjectType can be TABLE or INDEXr!r"N)rGetMessagesModuler FileReaderr read splitlinesParseSplitPointStringupper c_exceptionsInvalidArgumentExceptionformat SplitPointstableindexParseSplitValuekeysr expireTimeappend)rmsgssplit_points_listfilesingle_split_string single_splitsplits rProcesszSplitFileParser.ProcessLsf  ! !)T 2D   $** +t!%!7!7!9 112EF  - - -L)//19KK55228&9L2M     "  % + + - 8$\2%+ , ' - - /7 :$\2%+  %++L,FG%*  % %!77%   '5": ,8 9 ,8 s DE  E*c|jj|}|r3|jd|jd|jddStjddj |)a[Parses a string in the format " ()". and returns a dictionary with the extracted information. Args: input_string: The string to parse. Returns: A dictionary with keys "ObjectType", "ObjectName", and "SplitValue", or None if the input string is not in the expected format. )r rrr#r$)rmatchgroupr+r,r-)r input_stringr@s rr)z%SplitFileParser.ParseSplitPointStringnso  # # ) ), 7E  A A A  1 1  ..4f\.B  rctjdd}g}|j}|jj |s|j j |r%t jddj|g}|jj |}|rA|j|jd|j|jdnT|jj |}|r!|j|jdnt jdd|D]}|j}|jd}|j}|j|D]}tj } |d k(rd | _nn|d k(s|d k(s |d k(s|dk(rt%|j'| _n;|j+ddk(rt-|| _n|jd| _|j2j| |j||S)zParses a string in the format "(CommaSeparatedKeyParts) TableKey (CommaSeparatedKeyParts)". and returns a dictionary with the extracted information. Args: input_string: The string to parse. Returns: A split point key. rrr#zaInvalid split point string: {}. Each line must contain a single split point for a table or index.r=r>z2The split value must be surrounded by parenthesis.z()NULLTtruefalseTRUEFALSE')rr%striprr@rr+r,r-rr4rArKeyTokenizeWithCsvr JsonValueis_nullboollower boolean_valuefindfloat double_value string_valuekeyParts) rrBr5keys_allall_keys_stringsr@input_string_per_key single_key split_token key_partss rr1zSplitFileParser.ParseSplitValues  ! !)T 2DH%%'L 2288 4 4 : :< H  1 1  //5vl/C    ' ' - -l ;E ekk!n-ekk!n-%%++L9e  A/33  @  !11779177=88:j--.BC+))+ & ") V#'&'&*;+<+<+>&?I #%+',['9i$'2'8'8'>i$""9-#D$ooj!-!1. Orctjtj|ddtjd}t |S)zTokenizes text using commas as delimiters, ignoring commas within single quotes. Args: text: The text to tokenize. Returns: A list of tokens. rIT\) quotecharskipinitialspacequoting escapechar)csvreaderioStringIO QUOTE_NONEnext)rtextres rrMzSplitFileParser.TokenizeWithCsvs9ZZ Ds~~ F <rN) __name__ __module__ __qualname____doc__rr;r)r1rMrrr r s"> 8 D8>@rr c^t|j|jjS)r)r r rr;)argss rParseSplitPointsrrs$ ))4+E+E F N N PPr)rn __future__rrrrdrfrapitools.base.pyrgooglecloudsdk.api_lib.utilrgooglecloudsdk.callioperr+googlecloudsdk.core.utilr r rrrorrrxs<I&' (,>*zzzQr