-:j*RdZddlZddlmZmZddlZddlmZddlm Z m Z m Z ddl m Z mZddlmZdd lmZmZmZmZmZmZmZmZmZmZdd lmZdd lmZm Z e!eeZ"ee Z#ejHd e#Dcic]}||c}e% Z&ee%dzejNddjQe"dee"fZ)edZ*e*jWdgddddddeddddddf dee%dzejXdfdedededee&dzejNdfded e)d!ee-ejNd"d#d$fd%ee-ejNd&d'fd(ee-ejNd)d*d+fd,ed-ed.dfd/Z.e*jWd0gd1 dPd2ee%ejXd3fded-ed.dfd4Z/e*jWd5d6d7g dQd2ee%ejXd8fd,ed e)d-ed.df d9Z0e*jWd:gd; dQd2ee%ejXd8fdfd?ee%dzejNd@fd-ed.df dAZ1e*jWdBdCdDg dRdBee%ejXdEfd-ed.dfdFZ2e*jWdGgdH dSd2ee%ejXd8fdIee-ejNdJdKfdLee-ejNdMdNfd-ed.df dOZ3ycc}w)TaContains commands to interact with datasets on the Hugging Face Hub. Usage: # list datasets on the Hub hf datasets ls # list datasets with a search query hf datasets ls --search "code" # get info about a dataset hf datasets info HuggingFaceFW/fineweb N) Annotatedget_args)execute_raw_sql_query)CLIErrorRepositoryNotFoundErrorRevisionNotFoundError) DatasetSort_TExpandDatasetProperty_T) DatasetCard) REPO_LIST_DEFAULT_LIMIT AuthorOpt FilterOptLimitOpt RevisionOpt SearchOptTokenOpt get_hf_apimake_expand_properties_parser typer_factory)list_repo_files_cmd)_dataclass_to_dictoutDatasetSortEnum)typezComma-separated properties to return. When used, only the listed properties (and id) are returned. Example: '--expand=downloads,likes,tags'. Valid: z, .)helpcallbackz"Interact with datasets on the Hub.)rz list | ls)zhf datasets lsz*hf datasets ls --sort downloads --limit 10zhf datasets ls --search "code"z*hf datasets ls --filter benchmark:officialz$hf datasets ls HuggingFaceFW/finewebz'hf datasets ls HuggingFaceFW/fineweb -Rz.hf datasets ls HuggingFaceFW/fineweb --tree -h)examplesFrepo_idzVDataset ID (e.g. `username/repo-name`) to list files from. If omitted, lists datasets.searchauthorfiltersortz Sort results.limitexpandhuman_readablez--human-readablez-hz=Show sizes in human readable format (only for listing files).as_treez--treez3List files in tree format (only for listing files). recursivez --recursivez-Rz0List files recursively (only for listing files).revisiontokenreturnc ||tjd|tjd|tjd|tjd|tk7rtjd|tjdt|d||| | | S|rtjd | rtjd |rtjd | tjd t | } |r |j nd} | j |||| ||Dcgc] }t|}}tj|ycc}w)zList datasets on the Hub, or files in a dataset repo. When called with no argument, lists datasets on the Hub. When called with a dataset ID, lists files in that dataset repo. Nz'Cannot use --search when listing files.z'Cannot use --author when listing files.z'Cannot use --filter when listing files.z%Cannot use --sort when listing files.z&Cannot use --limit when listing files.z'Cannot use --expand when listing files.dataset)r repo_typer'r(r)r*r+z(Cannot use --tree when listing datasets.z-Cannot use --recursive when listing datasets.z2Cannot use --human-readable when listing datasets.z,Cannot use --revision when listing datasets.r+)r#r"r!r$r%r&) typer BadParameterr rrvalue list_datasetsrrtable)r r!r"r#r$r%r&r'r(r)r*r+apisort_key dataset_inforesultss ^/var/www/html/tokenscope/api/venv/lib/python3.12/site-packages/huggingface_hub/cli/datasets.py datasets_lsr;Fs\  $$%NO O  $$%NO O  $$%NO O  $$%LM M + +$$%MN N  $$%NO O")    !KLL  !PQQ  !UVV  !OPP 5 !C!tzztH -- .    <( G IIg s7E! leaderboard)z4hf datasets leaderboard SWE-bench/SWE-bench_VerifiedzLhf datasets leaderboard SWE-bench/SWE-bench_Verified --limit 5 --format jsonzIhf datasets ls --filter benchmark:official # list available leaderboards dataset_idz?The benchmark dataset ID (e.g. `SWE-bench/SWE-bench_Verified`).c0t|}|j|}|d|Dcgc] }t|}}tj|gddtj d|r'tj d|d j d yycc}w) zList model scores from a dataset leaderboard. This command helps find the best models for a task or compare models by benchmark scores. Use 'hf datasets ls --filter benchmark:official' to list available leaderboards.r0)r N)rankmodel_idr3sourcer@headersid_keyzPUse 'hf datasets ls --filter benchmark:official' to list available leaderboards.zUse 'hf models info rz' to get details about a model.)rget_dataset_leaderboardrrr5hintr@)r=r%r+r6r<entryr9s r:datasets_leaderboardrHs 5 !C--j-AK6A&56IJU!%(JGJII7 HH _` ' A(?(?'@@_`aKsBinfoz&hf datasets info HuggingFaceFW/finewebz9hf datasets info my-dataset --expand downloads,likes,tagsz+The dataset ID (e.g. `username/repo-name`).ct|} |j|||}t j |y#t$r}td|d|d}~wt$r}td|d|d|d}~wwxYw) z$Get info about a dataset on the Hub.r0)r r*r&z Dataset 'z ' not found.Nz Revision 'z' not found on 'z'.)rr8rrrrdict)r=r*r&r+r6rIes r: datasets_inforMs 5 !CU XfU HHTN #D:,l;rsZ  & A[[I0   /,H%<=>' $))-m/L1/LSVW $JELLdeienenoAeBdCCDE./AB  "FG    -    7O  d tu wO  O  O O $ /* , O O O  '4st vO$  X$YZ \%O,  ]D/ab d-O45O6 7O8 9O Odb#~u~~3tuuvb b b bb& 0C! #~u~~3`aab     " ptZ^ Q#~u~~3`aab Q cDj,%,,z@j"kk l Q S4Z3U!VV W Q  Q  Q Q   c j  3,GHH I       mrej g#~u~~3`aabglell<>ghhig D,%,,x6`aa bg  g  ggk0Ms> L$