H(dZddlmZddlmZddlmZddlmZddlmZddl m Z ddl m Z Gd d Z Gd d e Z Gd de ZGdde Zy)z1The different TableReader options for the BQ CLI.)absolute_import)division)print_function)Optional) discovery)bq_error) bq_id_utilsc eZdZdZ ddeedeedeefdZ ddeedeedeefdZdZ d efd Z d efd Z d efd Z ddeedeed eedeefdZ y) _TableReaderzBase class that defines the TableReader interface. _TableReaders provide a way to read paginated rows and schemas from a table. N start_rowmax_rowsselected_fieldsc4|j|||\}}|S)agRead at most max_rows rows from a table. Args: start_row: first row to return. max_rows: maximum number of rows to return. selected_fields: a subset of fields to return. Raises: BigqueryInterfaceError: when bigquery returns something unexpected. Returns: list of rows, each of which is a list of field values. )r r r)ReadSchemaAndRows)selfr r r_rowss #platform/bq/clients/table_reader.pyReadRowsz_TableReader.ReadRowss-&&&''IQ Kc| td| tdd}g}i}t||kr|t|z }t|ds td|jrt |j|}|j |rdn||||\}}} |s| r| jdg}|D](} |j|j|| |dz }*|r|s ||fSt||kr||fS) aRead at most max_rows rows from a table and the schema. Args: start_row: first row to read. max_rows: maximum number of rows to return. selected_fields: a subset of fields to return. Raises: BigqueryInterfaceError: when bigquery returns something unexpected. ValueError: when start_row is None. ValueError: when max_rows is None. Returns: A tuple where the first item is the list of fields and the second item a list of rows. Nzstart_row is requiredzmax_rows is requiredmax_rows_per_requestz9Subclass must have max_rows_per_request instance variable)r page_tokenrfields) ValueErrorlenhasattrNotImplementedErrorrmin _ReadOnePagegetappend_ConvertFromFV) rr r rrrschema rows_to_read more_rowscurrent_schemarows rrz_TableReader.ReadSchemaAndRows0s), . // - ..J D F d)h D )l T1 2! G    " "444lC 040A0A$)) 1B1-y*n ##Hb1# D''45Q y D>+ d)h * D>rc |sy|jdgDcgc]}|jdd}}g}t||D]1\}}d|vrtjd|dj dk(r|jdg}|jd d j d k(r@|j |D cgc]$} |j || jdd&c} |j |j |||jd d j d k(r1|j |D cgc]} | jddc} !|j |4|Scc}wcc} wcc} w) z;Converts from FV format to possibly nested lists of values.Nfvtypez'Invalid response: missing type propertyRECORDrmodeNULLABLEREPEATED)r"ziprBigqueryCommunicationErrorupperr#r$) rr%r)entryvaluesresultfieldr, subfieldssubvalues rr$z_TableReader._ConvertFromFVdsq  .1ggc2.> ?.>UeiiR .>F ? F'q u 11 5   v   ( *IIh+ 99VZ ( . . 0J > -- (!!)X\\#r-BC  --++Iq9 : 99VZ ( . . 0J > Q?Qx||C,Q?@  a-(. M3@ @sE8;)E= F returnc"|jSN_GetPrintContextrs r__str__z_TableReader.__str__  ""rc"|jSr>r?rAs r__repr__z_TableReader.__repr__rCrctd)z'Returns context for what is being read.z'Subclass must implement GetPrintContextrrAs rr@z_TableReader._GetPrintContexts G HHrrctd)aRead one page of data, up to max_rows rows. Assumes that the table is ready for reading. Will signal an error otherwise. Args: start_row: first row to read. max_rows: maximum number of rows to return. page_token: Optional. current page token. selected_fields: a subset of field to return. Returns: tuple of: rows: the actual rows of the table, in f,v format. page_token: the page token of the next page of results. schema: the schema of the table. z$Subclass must implement _ReadOnePagerG)rr r rrs rr!z_TableReader._ReadOnePages. D EEr)rNNr>NN)__name__ __module__ __qualname____doc__rintstrrrr$rBrEr@r!rrr r s"# $'+ #  } <(, 2#2 2 } 2h>#s###II#''+ F#F F3- F  } Frr c eZdZdZdej dedejjfdZ de fdZ dd e ed e ed e e d e e fd Zy)TableTableReaderz&A TableReader that reads from a table.local_apiclientr table_refc.||_||_||_yr>)rTr _apiclient)rrSrrTs r__init__zTableTableReader.__init__s DN 4D%DOrr<c|jSr>)rTrAs rr@z!TableTableReader._GetPrintContexts >>#rNr r rrct|j}||d<|r||d<n||d<d}|||d<|8|jjjdi|j }|j dd}|j dg}t|j}|||d<|jjj di|j }|j di} ||| fS)N maxResults pageToken startIndexselectedFieldsrr%rP)dictrTrV tabledatalistexecuter"tables) rr r rrkwdsdatar table_infor%s rr!zTableTableReader._ReadOnePages  D!D$d;$d< D".d  | -T__ & & ( - - 5 5 = = ?d+t,J 88FB D  D".d -'')--55==?J ^^Hb )F *f %%rrI)rJrKrLrMrResourcerNr ApiClientHelperTableReferencerWrOr@rr!rPrrrRrRs.& ))& &,,;; &$$#''+ &#& &3- &  } &rrRc eZdZdZdej dedejjfdZ de fdZ dd e ed e ed e e d e e fd Zy)JobTableReaderz.A TableReader that reads from a completed job.rSrjob_refc.||_||_||_yr>)rkrrV)rrSrrks rrWzJobTableReader.__init__s DL 4D%DOrr<c|jSr>rkrAs rr@zJobTableReader._GetPrintContext <<!rNr r rrc|t|j}||d<d|d<|r||d<n||d<|jjjd i|j }|dst jd|d|jdd}|jd d}|jd g}|||fS) NrZr timeoutMsr[r\ jobCompleteJob is not doner%rrP) r^rkrVjobsgetQueryResultsrar BigqueryErrorr") rr r rrrcrdr%rs rr!zJobTableReader._ReadOnePages  D!DD$d;$d< 14??   ! 1 1 9D 9 A A CD    " "4#A BB+t,J XXh %F 88FB D *f %%rrIrJrKrLrMrrfrNr rg JobReferencerWrOr@rr!rPrrrjrjs6& ))& &**77 &""#''+ &#& &3- &  } &rrjc eZdZdZdej dedejjfdZ de fdZ dd e ed e ed e e d e e fd Zy)QueryTableReaderz0A TableReader that reads from a completed query.rSrrkc<||_||_||_||_yr>)rkrrV_results)rrSrrkresultss rrWzQueryTableReader.__init__s!DL 4D%DODMrr<c|jSr>rnrAs rr@z!QueryTableReader._GetPrintContext rorNr r rrc|jrt|jni}||d<d|d<|r||d<n||d<|jdstjd|d|jj d d}|jj d d}|jj d d}|L|Jt |dk(r<|jj d d} |jj d g} d}n=|||t|tt |||zk\r|jj dd}t|t |kr|tjd |d|jj d d} |jj d g} n|jjjdi|j} | dstjd|d| j dd}| j d d} | j d g} | || fS)NrZrrqr[r\rrrsrtr totalRows jobReferencer%zSynchronous query z< did not return all rows, yet it did not return a page tokenrP) rkr^r}rrwr"rNrr rVrurvra) rr r rrrc result_rows total_rows job_referencer%rrds rr!zQueryTableReader._ReadOnePages"&4 2D!DD$d;$d< == '  " "4#A BB--##FD1K"";5JMM%%nd;M3z?a#7}}  40f ]]  vr *dj  #  !  C OY5I J J==$$[$7j [ C O + 0B$$&* -  }}  40f ]]  vr *d 3T__ ! ! # 3 3 ;d ; C C Ed - $$T%CDD88K.jxx$'f XXfb !d *f %%rrIrxrPrrr{r{s8  ))   **77 ""#''+ 3&#3& 3&3- 3&  } 3&rr{N)rM __future__rrrtypingrgoogleapiclientrutilsrr r rRrjr{rPrrrsT7&%%TFTFn+&|+&\%&\%&PE&|E&r