Server : LiteSpeed System : Linux barito.iixcp.rumahweb.net 5.14.0-611.49.1.el9_7.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Apr 21 16:39:08 EDT 2026 x86_64 User : elvh3918 ( 1528) PHP Version : 8.2.31 Disable Function : mail Directory : /usr/local/lib/python3.9/site-packages/celery/backends/__pycache__/ |
a
��Sh{0 � @ sp d Z ddlmZ ddlmZmZ ddlmZ ddlmZ ddl m
Z
ddl m Z ddl
mZ dd lmZ dd
lmZmZ dd lmZmZ dd lmZ d
dlmZ zTddlZddl m Z ddl m Z ddl m!Z! ddl"m#Z# ddl$m%Z% ddl&m'Z' W n e(�y dZ#Y n0 zddl"m)Z)m*Z* W n e(�y> dZ)dZ*Y n0 dZ+ee,�Z-G dd� de�Z.G dd� de.�Z/dS ) z5Google Cloud Storage result store backend for Celery.� )�ThreadPoolExecutor)�datetime� timedelta)�getpid)�RLock)� bytes_to_str)�
dictfilter)� url_to_parts)�maybe_signature)�
ChordError�ImproperlyConfigured)� GroupResult�allow_join_result)�
get_logger� )�KeyValueStoreBackendN��retry)�Conflict)�if_exception_type)�storage)�Client)�
DEFAULT_RETRY)� firestore�firestore_admin_v1)�
GCSBackendc sp e Zd ZdZ� fdd�Zdd� Zdd� Zdd � Zd
d � Ze d d
� �Z
e dd� �Z dd� Z dd� Z
dd� Z� ZS )�GCSBackendBasez)Google Cloud Storage task result backend.c s t s td��t� jf i |�� t� | _t� | _t| _ d | _
| j j }| j
rd| �� }|jf i t|��� |�d�| _| js~td��|�d�| _| js�td��|�dd��d�| _t|�d d
��| _t|�d �p�d �| _| jd k r�td
| j� d���n"| j�r| �� �std| j� ���d S )Nz8You must install google-cloud-storage to use gcs backend�
gcs_bucketz:Missing bucket name: specify gcs_bucket to use gcs backendZ gcs_projectz6Missing project:specify gcs_project to use gcs backend�
gcs_base_path� �/Zgcs_threadpool_maxsize�
Zgcs_ttlr z
Invalid ttl: z# must be greater than or equal to 0z>Missing lifecycle rule to use gcs backend with ttl on bucket: )r r �super�__init__r � _client_lockr �_pidr �
_retry_policy�_client�app�conf�url�_params_from_url�updater �get� bucket_name�project�strip� base_path�int�_threadpool_maxsize�float�ttl� _is_bucket_lifecycle_rule_exists)�self�kwargsr) Z
url_params�� __class__� �=/usr/local/lib/python3.9/site-packages/celery/backends/gcs.pyr# - sJ � � �
�
��zGCSBackendBase.__init__c C s@ t |�}| �|�}z|j| jd�W S tjjy: Y d S 0 d S �Nr )r � _get_blobZdownload_as_bytesr& r �blobZNotFound�r7 �keyr? r; r; r< r- U s
zGCSBackendBase.getc C sB t |�}| �|�}| jr.t�� t| jd� |_|j|| jd� d S )N��secondsr ) r r> r5 r �utcnowr Z custom_timeZupload_from_stringr& )r7 rA �valuer? r; r; r< �set] s
zGCSBackendBase.setc C s, t |�}| �|�}|�� r(|j| jd� d S r= )r r> �exists�deleter& r@ r; r; r< rH d s
zGCSBackendBase.deletec C s< t � �"}t|�| j|��W d � S 1 s.0 Y d S �N)r �list�mapr- )r7 �keys�poolr; r; r< �mgetj s zGCSBackendBase.mgetc C s� | j �� | jr.| jt� kr.| jW d � S t| jd�| _t� | _tjj| j | j dd�}| jj
}|� d|� |j j
� d|� | jW d � S 1 s�0 Y dS )zReturns a storage client.N�r/ � )�pool_connections� pool_maxsize� max_retrieszhttps://)r$ r' r% r r r/ �requests�adapters� HTTPAdapterr3 Z_http�mountZ
_auth_request�session)r7 �adapterZ client_httpr; r; r< �clientn s � zGCSBackendBase.clientc C s | j �| j�S rI )rZ �bucketr. �r7 r; r; r< r[ � s zGCSBackendBase.bucketc C s& | j r| j � d|� �n|}| j�|�S )Nr )r1 r[ r? )r7 rA Zkey_bucket_pathr; r; r< r> � s zGCSBackendBase._get_blobc C s4 | j }|�� |jD ]}|d d dkr dS qdS )N�action�type�DeleteTF)r[ �reloadZlifecycle_rules)r7 r[ Zruler; r; r< r6 � s
z/GCSBackendBase._is_bucket_lifecycle_rule_existsc C s t | j�}|j|jd�|j�S )N)r r )r r* �hostname�path�query)r7 Z url_partsr; r; r< r+ � s
��z GCSBackendBase._params_from_url)�__name__�
__module__� __qualname__�__doc__r# r- rF rH rN �propertyrZ r[ r> r6 r+ �
__classcell__r; r; r9 r< r * s (
r c s� e Zd ZdZdZdZdZdZdZ� fdd�Z e
dd � �Z d
d � Z � fd d
�Z
eed�dd�Zdd� Zdd� Zdd� Zdd� Zdd� Z� ZS )r zWGoogle Cloud Storage task result backend.
Uses Firestore for chord ref count.
TZceleryZ chord_countZ
expires_atc sh t rtstd��t� jf i |�� t� | _d | _| jj �
d| j �| _ | �
� sdtd| j� d| j � ���d S )Nz:You must install google-cloud-firestore to use gcs backend�firestore_projectzHMissing TTL policy to use gcs backend with ttl on Firestore collection: z
project: )r r r r" r# r �_firestore_lock�_firestore_clientr( r) r- r/ rj � _is_firestore_ttl_policy_enabled�_collection_name)r7 r8 r9 r; r< r# � s$ �����zGCSBackend.__init__c C sj | j �N | jr.| jt� kr.| jW d � S tj| jd�| _t� | _W d � n1 sZ0 Y | jS )zReturns a firestore client.NrO )rk rl r% r r r rj r\ r; r; r< �firestore_client� s �&zGCSBackend.firestore_clientc C sd t �� }d| j� d| j� d| j� �}t j|d�}|j|d�}|j}|ob|jt j j
j j t j j
j j
hv S )Nz projects/z&/databases/(default)/collectionGroups/z/fields/)�name)�request)r ZFirestoreAdminClientrj rn �_field_expiresZGetFieldRequest� get_field�
ttl_config�stateZFieldZ TtlConfig�StateZACTIVEZCREATING)r7 rZ rp rq �fieldrt r; r; r< rm � s
���
�z+GCSBackend._is_firestore_ttl_policy_enabledc s4 | � |d ��� }| �|d� t� j||fi |��S )Nr i�Q )�get_key_for_chord�decode�_expire_chord_keyr" �_apply_chord_incr)r7 Zheader_result_args�bodyr8 rA r9 r; r< r{ � s z GCSBackend._apply_chord_incr)rA �returnc
C sF | � |�}|j| jt�d�idtjtt�ddddd�d�}|j d j
S ) Nr Tg �?g �f@g @)� predicate�initial�maximum�
multiplier�timeout)�merger r ) �_firestore_documentrF � _field_countr Z Incrementr �Retryr r Ztransform_resultsZ
integer_value)r7 rA �doc�respr; r; r< �incr� s
�� zGCSBackend.incrc K s� | j }|j}|sdS | �|�}| �|�}|j�d�} | du r\| �||�}
|
du rTdS t|
�} || krtt� d|� �nx|| k�r�| �||�}
|
du r�dS t
|j|d�} |
j } �z*z:t � � | |j
jdd�}
W d � n1 s�0 Y W n~ t�yf } zdz t|
�� �}d�||�}W n t�y2 t|�}Y n0 t�d||� | �| t|�� W Y d}~ndd}~0 0 z| �|
� W nJ t�y� } z0t�d||� | �| td |���� W Y d}~n
d}~0 0 W |
�� | �|� n|
�� | �|� 0 dS )
a~ Chord part return callback.
Called for each task in the chord.
Increments the counter stored in Firestore.
If the counter reaches the number of tasks in the chord, the callback
is called.
If the callback raises an exception, the chord is marked as errored.
If the callback returns a value, the chord is marked as successful.
NZ
chord_sizez/Chord counter incremented too many times for %r�r( T)r� � propagatez Dependency {0.id} raised {1!r}�Chord %r raised: %rzCallback error: )r( �grouprx r� �chordr- �
_restore_deps�len�logger�warningr
Z join_nativer r) Zresult_chord_join_timeout� Exception�nextZ_failed_join_report�format�
StopIteration�repr� exception�chord_error_from_stackr �delayrH �_delete_chord_key)r7 rq ru �resultr8 r( �gidrA �val�size�deps�callback�j�ret�excZculprit�reasonr; r; r< �on_chord_part_return� sf
�
�( �& � �z GCSBackend.on_chord_part_returnc
C s� | j }ztj|| d�}W nX typ } z@t|j|d�}t�d||� | �|t d|���� W Y d }~d S d }~0 0 |d u r�z t
|��W nX t
y� } z@t|j|d�}t�d||� | �|t d|� d��� W Y d }~n
d }~0 0 |S )N)�backendr� r� zCannot restore group: z Chord callback %r raised: %rz GroupResult z no longer exists) r( r
�restorer� r
r� r� r� r� r �
ValueError)r7 r� rq r( r� r� r� r; r; r< r� 7 s, � �zGCSBackend._restore_depsc C s | � |�}|�� d S rI )r� rH )r7 rA r� r; r; r< r� O s
z GCSBackend._delete_chord_keyc C s4 t �� t|d� }| �|�}|j| j|idd� dS )z�Set TTL policy for a Firestore document.
Firestore ttl data is typically deleted within 24 hours after its
expiration date.
rB T)r� N)r rD r r� rF rr )r7 rA �expiresZ val_expiresr� r; r; r< rz S s
z GCSBackend._expire_chord_keyc C s | j �| j��t|��S rI )ro Z
collectionrn �documentr )r7 rA r; r; r< r� ] s
��z GCSBackend._firestore_document)rd re rf rg Zimplements_incrZsupports_native_joinrn r� rr r# rh ro rm r{ �bytesr2 r� r� r� r� rz r� ri r; r; r9 r< r � s"
A
r )0rg �concurrent.futuresr r r �osr � threadingr Zkombu.utils.encodingr Zkombu.utils.functionalr Zkombu.utils.urlr Z
celery.canvasr
Zcelery.exceptionsr r Z
celery.resultr
r Zcelery.utils.logr �baser rT Zgoogle.api_corer Zgoogle.api_core.exceptionsr Zgoogle.api_core.retryr Z google.cloudr Zgoogle.cloud.storager Zgoogle.cloud.storage.retryr � ImportErrorr r �__all__rd r� r r r; r; r; r< �<module> s>
v