) : d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlZddlZddlZddl m
Z
ddl m Z dd l
mZ dd
lmZ dd lmZ dd lmZ dd
lmZ ddlmZ ddlmZ ddlmZ ddlmZ ddl m Z ddl m Z ddl m Z ddl!m"Z" ddlm#Z# dZ$de$z dz Z% G d de Z&y) zCImplementation of Unix-like du command for cloud storage providers. )absolute_import)print_function)division)unicode_literalsN)BucketListingObject)Command)CommandArgument) ApiSelector)CommandException)ContainsWildcard)StorageUrlFromString) ls_helper)NO_MAX)S3_DELETE_MARKER_GUID)UTF8)GcloudStorageFlag)GcloudStorageMap) print_to_fd)MakeHumanReadable) text_utilz
gsutil du url...
z
SYNOPSIS
ak
DESCRIPTION
The du command displays the amount of space in bytes used up by the
objects in a bucket, subdirectory, or project. The syntax emulates
the Linux ``du -b`` command, which reports the disk usage of files and subdirectories.
For example, the following command reports the total space used by all objects and
subdirectories under gs://your-bucket/dir:
gsutil du -s -a gs://your-bucket/dir
OPTIONS
-0 Ends each output line with a 0 byte rather than a newline. You
can use this to make the output machine-readable.
-a Includes both live and noncurrent object versions. Also prints the
generation and metageneration number for each listed object. If
this flag is not specified, only live object versions are included.
-c Includes a total size at the end of the output.
-e Exclude a pattern from the report. Example: -e "*.o"
excludes any object that ends in ".o". Can be specified multiple
times.
-h Prints object sizes in human-readable format. For example, ``1 KiB``,
``234 MiB``, or ``2GiB``.
-s Displays only the total size for each argument, omitting the list of
individual objects.
-X Similar to ``-e``, but excludes patterns from the given file. The
patterns to exclude should be listed one per line.
EXAMPLES
To list the size of each object in a bucket:
gsutil du gs://bucketname
To list the size of each object in the ``prefix`` subdirectory:
gsutil du gs://bucketname/prefix/*
To include the total number of bytes in human-readable form:
gsutil du -ch gs://bucketname
To see only the summary of the total number of (live) bytes in two given
buckets:
gsutil du -s gs://bucket1 gs://bucket2
To list the size of each object in a bucket with `Object Versioning
`_ enabled,
including noncurrent objects:
gsutil du -a gs://bucketname
To list the size of each object in a bucket, except objects that end in ".bak",
with each object printed ending in a null byte:
gsutil du -e "*.bak" -0 gs://bucketname
To list the size of each bucket in a project and the total size of the
project:
gsutil -o GSUtil:default_project_id=project-name du -shc
c j e Zd ZdZ ej
dg ededdddej ej gej e j g Z
ej dg dd ei
Z ed dg ed ed
ed ed ed ed ed d Zd Zd Zd Zy) DuCommandz$Implementation of gsutil du command.dur z 0ace:hsX:FT) command_name_aliasesusage_synopsismin_argsmax_argssupported_sub_args file_url_okprovider_url_okurls_start_arggs_api_supportgs_default_apiargparse_arguments command_helpzDisplay object size usage) help_namehelp_name_aliases help_typehelp_one_line_summary help_textsubcommand_help_textstoragez--zero-terminatorz--all-versionsz--totalz--exclude-name-patternz--readable-sizesz --summarizez--exclude-name-pattern-file)-0-a-c-e-h-s-X)gcloud_commandflag_mapc | j r t | nt j | }t j
dj
|t j | | j y )Nz{size:<11} {name})sizename)end) human_readabler six text_typer r format ensure_text line_ending)self num_bytesr8 size_strings $platform/gsutil/gslib/commands/du.py_PrintSummaryLinezDuCommand._PrintSummaryLine s^ )) %Y//2}}Y/G
.55
st 4 6 6 " . .0 c |j }|j }|j r(t |j j v r d}d}d}|dz
}nD| j
rt
|j nt |j }|j }d}| j s`dj |t j | t j | j }t |t j d ||fS ) zPrint listing info for given bucket_listing_ref.
Args:
bucket_listing_ref: BucketListing being listed.
Returns:
Tuple (number of objects, object size)
Raises:
Exception: if calling bug encountered.
0r z z{size:<11} {url}{ending})r7 urlending )filer9 ) root_object
url_stringmetadatar additionalPropertiesr: r r7 str summary_onlyr= r; r> r? r sysstdout)r@ bucket_listing_refobjurl_strrB rA num_objs
url_details rC _PrintInfoAboutBucketListingRefz)DuCommand._PrintInfoAboutBucketListingRef s
(
(C ++G !B!BBkih
!!g ++ 'sxx014SXX ((ih .55oog& !1!12 6 4j *3::26 i rE c H d _ d _ d _ d _ d _ g _ j r j D ] \ }}|dk( rd _ |dk( rd _ |dk( rd _ -|dk( r j
j | N|d k( rd _ [|d
k( rd _ h|d k( sn|d k( rt j }d}n0t j r t |d
nt |d
t }d}|D cg c]% }t j |j ' c} _ |s|j! j" sdg _ d}d} fd}d d} fd}
j" D ] } t% | } | j' rt) d j* z dg}
t- j. j0 j2 || | |
| j d j
|
}| j5 r0 j1 d| j6 z j9 dg }nX| j; r< j1 | j6 d| j< j9 dg }n t? | g}|D ] }|j@ }|j; r' j rt% |jC d }|jE | \ }}}|jG r |dk( rtI | r j
sd}||z
} j s jK ||jL jO d jQ t j r jK |d |r t) d yc c}w )!z'Command entry point for the du command.
Fr- r. Tr/ r0 r1 r2 r3 -r)encodingzgs://r c & j | S N)rZ )blrr@ s rC _PrintObjectLongz.DuCommand.RunCommand.._PrintObjectLong s
1
1#
66rE c y rb )
unused_blrs rC
_PrintNothingz+DuCommand.RunCommand.._PrintNothing s
rE c ~ j s0j | |j j t y y rb )rR rD rN encoder )rA rc r@ s rC _PrintDirectoryz-DuCommand.RunCommand.._PrintDirectory s0
y#..*?*?*EF rE z$Only cloud URLs are supported for %sr7 ) print_object_funcprint_dir_funcprint_dir_header_funcprint_dir_summary_funcprint_newline_func all_versionsshould_recurseexclude_patternsfieldsz%s://*id)
bucket_fieldsz://z**)wildcard_suffix/totalz$One or more URLs matched no objects.rb ))r? rq
produce_totalr: rR rs sub_optsappendrS stdinr; PY2openr r> stripcloseargsr
IsFileUrlr command_namer LsHelperWildcardIteratorlogger
IsProviderscheme IterBucketsIsBucket bucket_namer storage_urlCreatePrefixUrlExpandUrlAndPrintIsObjectr rD rN rstriprj )r@ oaff_closeline total_bytesgot_nomatch_errorsrd rh rk url_argtop_level_storage_urlbucket_listing_fieldslisting_helpertop_level_iterrc r _exp_objs exp_bytess ` rC
RunCommandzDuCommand.RunCommand sx D D D D D D }}--$!Q 9 !$
$Y "$
$Y #$
$Y
&
&q
)
$Y $$
$Y "$
$Y
#X AG #Q T!S4-HAGMN"OQT3??4::<#@Q"O$
GGI- 0 99)diK 7 G 99 27 ; ( ( * E # 0 0 1 2 2 %h ))
++ ,& -!0 *(( 00& (n
) ) + .. ! ( (
)**5+TF+*K ! ) ) + ...55.::
<==H[,06 >I >3
..CDE #oo !d&7&7,))$)?A+!/!A!A+!N 8Y "x1} W %d.C.C #
y
!$ !6!6s!;!B!B4!H J G d
['2 C DD [ #Ps ?*N N)__name__
__module__ __qualname____doc__r CreateCommandSpec _SYNOPSISr r
XMLJSONr MakeZeroOrMoreCloudURLsArgument command_specHelpSpec_DETAILED_HELP_TEXT help_specr r gcloud_storage_maprD rZ r rf rE rC r r z s , +**
$!oo{'7'78 %%
9/
9
9
;," g 7#
) ( &!"56!"23!),!":;!"45!-0!"?@ 0!!Fj
rE r )'r
__future__r r r r localerS r; gslib.bucket_listing_refr
gslib.commandr gslib.command_argumentr gslib.cs_api_mapr
gslib.exceptionr gslib.storage_urlr r
gslib.utilsr gslib.utils.constantsr r r gslib.utils.shim_utilr r gslib.utils.text_utilr gslib.utils.unit_utilr r r r r rf rE rC r s J & % '
8 ! 2 ( , . 2 ! ( 7 & 3 2 - 3 !
FH V
rE