File: //opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/etcd_util.cpython-310.pyc
o
�N�g߄ � @ s d Z ddlZddlZddlZddlmZ zddlZddlm Z m
Z
dZ W n e y/ dZ Y nw zddl
Z
dZW n e yC dZY nw e�e�ZG dd� de�ZG d d
� d
e�ZG d d � d e�ZG d
d� de�ZG dd� d�ZG dd� de�ZG dd� de�Zddd�Zddd�Zdd� Z dS ) a�
Utilities for working with etcd
.. versionadded:: 2014.7.0
:depends: - python-etcd or etcd3-py
This library sets up a client object for etcd, using the configuration passed
into the get_conn() function. Normally, this is __opts__. Optionally, a profile
may be passed in. The following configurations are both valid:
.. code-block:: yaml
# No profile name
etcd.host: 127.0.0.1
etcd.port: 2379
etcd.username: larry # Optional; requires etcd.password to be set
etcd.password: 123pass # Optional; requires etcd.username to be set
etcd.ca: /path/to/your/ca_cert/ca.pem # Optional
etcd.client_key: /path/to/your/client_key/client-key.pem # Optional; requires etcd.ca and etcd.client_cert to be set
etcd.client_cert: /path/to/your/client_cert/client.pem # Optional; requires etcd.ca and etcd.client_key to be set
etcd.require_v2: True # Optional; defaults to True
etcd.encode_keys: False # Optional (v3 ONLY); defaults to False
etcd.encode_values: True # Optional (v3 ONLY); defaults to True
etcd.raw_keys: False # Optional (v3 ONLY); defaults to False
etcd.raw_values: False # Optional (v3 ONLY); defaults to False
etcd.unicode_errors: "surrogateescape" # Optional (v3 ONLY); defaults to "surrogateescape"
# One or more profiles defined
my_etcd_config:
etcd.host: 127.0.0.1
etcd.port: 2379
etcd.username: larry # Optional; requires etcd.password to be set
etcd.password: 123pass # Optional; requires etcd.username to be set
etcd.ca: /path/to/your/ca_cert/ca.pem # Optional
etcd.client_key: /path/to/your/client_key/client-key.pem # Optional; requires etcd.ca and etcd.client_cert to be set
etcd.client_cert: /path/to/your/client_cert/client.pem # Optional; requires etcd.ca and etcd.client_key to be set
etcd.require_v2: True # Optional; defaults to True
etcd.encode_keys: False # Optional (v3 ONLY); defaults to False
etcd.encode_values: True # Optional (v3 ONLY); defaults to True
etcd.raw_keys: False # Optional (v3 ONLY); defaults to False
etcd.raw_values: False # Optional (v3 ONLY); defaults to False
etcd.unicode_errors: "surrogateescape" # Optional (v3 ONLY); defaults to "surrogateescape"
Encoding keys for etcd v3 allows a differentiation within etcd between byte and string keys.
It is worth noting that if you chose to encode keys, due to the way encoding pre-etcd with msgpack works,
all recursive functionality will not work as intended. This includes tree and ls along
with all methods that have recurse kwargs. Thus, enabling this option is not recommended.
Once configured, the client() function is passed a set of opts, and optionally,
the name of a profile to be used.
.. code-block:: python
import salt.utils.etcd_utils
client = salt.utils.etcd_utils.get_conn(__opts__, profile='my_etcd_config')
You may also use the newer syntax and bypass the generator function.
V2 API
.. code-block:: python
import salt.utils.etcd_utils
client = salt.utils.etcd_utils.EtcdClient(__opts__, profile='my_etcd_config')
V3 API
.. versionadded:: 3005
.. code-block:: python
import salt.utils.etcd_utils
client = salt.utils.etcd_utils.EtcdClientV3(__opts__, profile='my_etcd_config')
It should be noted that some usages of etcd require a profile to be specified,
rather than top-level configurations. This being the case, it is better to
always use a named configuration profile, as shown above.
� N)�
SaltException)�
MaxRetryError�ReadTimeoutErrorTFc @ � e Zd ZdZdS )�EtcdLibraryNotInstalled�2
We didn't find the required etcd library
N��__name__�
__module__� __qualname__�__doc__� r
r
�H/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/etcd_util.pyr h � r c @ r )� IncompatibleEtcdRequirementszX
A user is explicitly creating a client class, but requires a different version
Nr r
r
r
r r n r r c @ r )�Etcd3DirectoryExceptionr Nr r
r
r
r r t r r c @ r )�EtcdUtilWatchTimeoutz6
A watch timed out without returning a result
Nr r
r
r
r r z r r c @ s� e Zd ZdZ d"dd�Zd#dd�Zd$d d
�Z d%d d �Zd&dd�Zd&dd�Z d'dd�Z
d'dd�Z d(dd�Z d(dd�Z
dd� Zd$d d �Zd$d d �Zd d!� ZdS ))�EtcdBasez�
Base class for the different versions of etcd clients.
This also serves as a documentation hub for all superclasses.
Nc
K s2 |
� dd�s t||�} n|} | | _|p| j� dd�| _|p#| j� dd�| _| jdkr4| jdkr4t�d� |p;| j� d�}|pC| j� d �}|pK| j� d
�} |pS| j� d �}
| p[| j� d �}i }|rk|rkt|�t|�d
�}i }| rz|sz|
szt| �dd�}| r�|r�|
r�||
f}t| �|dd�}|�� | _ | j �
|� d S )N�
has_etcd_optsFz etcd.hostz 127.0.0.1z etcd.portiK z:Using default etcd host and port, use a profile if needed.z
etcd.usernamez
etcd.passwordzetcd.cazetcd.client_keyzetcd.client_cert)�username�password�https)�ca_cert�protocol)r �certr ) �get�_get_etcd_opts�conf�host�port�log�warning�str�copy�xargs�update)�self�opts�profiler r r r �caZ
client_keyZ client_cert�kwargsZ etcd_optsr Zcli_keyZcli_certZauth�certsr r
r
r �__init__� s:
� �
zEtcdBase.__init__Fr c K � t � ��N��NotImplementedError)r& �key�recurse�timeout�start_revisionr* r
r
r �watch� s zEtcdBase.watchc C r- )zi
Get the value of a specific key. If recurse is true, defer to EtcdBase.tree() instead.
r/ �r& r1 r2 r
r
r r � � z EtcdBase.getc K r- )z�
Read a value of a key.
This method also provides the ability to wait for changes after a given index and/or
within a certain timeout.
r/ )r& r1 r2 �waitr3 r4 r* r
r
r �read� s z
EtcdBase.read� c C s� |s|i iS |� d�}i }|�� D ].\}}|� d�}|r%d|� d|� �}nd|� �}t|t�r;| �||�}|�|� q|||< q|S )a%
Take a data dictionary and flatten it to a dictionary with values that are all strings.
If path is given, prepend it to all keys.
For example, given path="/salt" it will convert...
{
"key1": "value1",
"key2": {
"subkey1": "subvalue1",
"subkey2": "subvalue2",
}
}
to...
{
"/salt/key1": "value1",
"/salt/key2/subkey1": "subvalue1",
"/salt/key2/subkey2": "subvalue2",
}
�/)�strip�items�
isinstance�dict�_flattenr% )r& �data�pathZflat�k�v�p�retr
r
r r@ � s
zEtcdBase._flattenc C r- )a�
Update etcd according to the layout of fields.
Given etcd with this layout...
{
...
"/salt/key1": "OLDvalue1",
"/salt/key2/subkey1": "OLDsubvalue1",
"/salt/key2/subkey2": "OLDsubvalue2",
...
}
fields = {
"key1": "value1",
"key2": {
"subkey1": "subvalue1",
"subkey2": "subvalue2",
}
}
will update etcd to look like the following...
{
...
"/salt/key1": "value1",
"/salt/key2/subkey1": "subvalue1",
"/salt/key2/subkey2": "subvalue2",
...
}
r/ )r& �fieldsrB r
r
r r% � s zEtcdBase.updatec C s | j ||||d�S )zH
Write a file or directory, a higher interface to write
)�ttl� directory)�write�r& r1 �valuerH rI r
r
r �set s z EtcdBase.setc C s |r | � |||�S | �|||�S )�G
Write a file or directory depending on directory flag
)�write_directory�
write_filerK r
r
r rJ & s zEtcdBase.writec C r- )z8
Write a file (key: value pair) to etcd
r/ �r& r1 rL rH r
r
r rP . r7 zEtcdBase.write_filec C r- )z5
Write a directory (key: {}) to etcd
r/ rQ r
r
r rO 4 r7 zEtcdBase.write_directoryc C r- )z�
Get all the top level keys and their values at the given path.
If the key is a directory, its value is an empty dictionary.
r/ �r& rB r
r
r �ls: � z EtcdBase.lsc C s | � ||�S )z%
An alias for delete
)�deleter6 r
r
r �rmB s z EtcdBase.rmc K r- )z@
Delete keys or (recursively) whole directories
r/ )r& r1 r2 r* r
r
r rU H r7 zEtcdBase.deletec C r- )z`
.. versionadded:: 2014.7.0
Recurse through etcd and return all values
r/ rR r
r
r �treeN rT z
EtcdBase.tree)NNNNNNNN�Fr N�F�FFNN�r: �NFr. )r r
r r r, r5 r r9 r@ r% rM rJ rP rO rS rV rU rW r
r
r
r r � s8
�
5
�
)
!
r c s� e Zd Z� fdd�Zddd�Zd dd �Z d d
d �Zd d
d�Zd dd�Zd dd�Z d dd�Z
dd� Z d dd�Z dd� Z
� ZS )!�
EtcdClientc sb t std��t�d� t� j|fi |�� | j�dd�s td��t j
d| j | j d�| j
��| _d S )Nz+Don't have python-etcd, need to install it.z.etcd_util has the libraries needed for etcd v2�etcd.require_v2Tz%Can't create v2 with a v3 requirement�r r r
)� HAS_ETCD_V2r r �debug�superr, r r r �etcd�Clientr r r$ �client)r& r' r* �� __class__r
r r, X s
"zEtcdClient.__init__Fr Nc
K s� |� dd �}|d urtjj�dd� |}|r t�d|� |d dddd�}z | j||d||d �}W n~ ty{ z| �|�}W n t j
yQ t� d
� | Y Y S t y] i Y Y S w |rit
|d �rid|d < t
|d �|d < t
|d
�|d< | Y S ty� t�d|� i Y S t jy� } z
t�d| � W Y d } ~ d S d } ~ w t y� i Y S w |d u r�i S |r�t
|dd �|d< t
|d d �|d < t
|d d �|d < d|d< t
|d
�|d< |S )N�index�Argon�qThe index kwarg has been deprecated, and will be removed in the Argon release. Please use start_revision instead.�-Invalid kwargs passed in will not be used: %sFr �r1 rL �changed�mIndex�dirT�r2 r8 r3 r4 z(etcd: key was not created while watchingro rL Z
modifiedIndexrn zKetcd: failed to perform 'watch' operation on key %s due to connection error�etcd: %sr1 rm )�pop�salt�utils�versions�
warn_untilr r! r9 r rc �EtcdKeyNotFoundra �
ValueError�getattrr �error�EtcdConnectionFailed)
r& r1 r2 r3 r4 r* rh rF �result�errr
r
r r5 d sl �
�
� � � �zEtcdClient.watchc C sr |s4z| � |�}W n$ tjy Y d S tjy$ t�d|� Y d S ty- Y d S w t|dd �S | �|�S )NzIetcd: failed to perform 'get' operation on key %s due to connection errorrL ) r9 rc rw r{ r rz rx ry rW �r& r1 r2 r| r
r
r r � s � �
zEtcdClient.getc
K s� |� dd �}|� dd �}|d urtjj�dd� |}|d ur(tjj�dd� |}|r0t�d|� z |rA| jj|||||d�} W | S | jj||||d�} W | S t j
t j fyd }
zt� d |
� � d }
~
w t
y� |ryt�d
� td |� d ���t� d
� t �
d�� ty� }
z
t� d� t �
d��d }
~
w t jy� }
zt� d |
� � d }
~
w ty� t� d� � ty� }
zt� d|
� � d }
~
ww )N� recursive� waitIndexri �nThe recursive kwarg has been deprecated, and will be removed in the Argon release. Please use recurse instead.�uThe waitIndex kwarg has been deprecated, and will be removed in the Argon release. Please use start_revision instead.rk )r r8 r3 r� )r r8 r3 rq z&etcd: Timed out while executing a waitz Watch on z
timed outzetcd: Timed outzConnection failedzetcd: Could not connectz Could not connect to etcd serverzZetcd: error. python-etcd does not fully support python 2.6, no error information available�etcd: uncaught exception %s)rr rs rt ru rv r r! re r9 rc r{ rw rz r ra r r Z
EtcdExceptionrx � Exception) r& r1 r2 r8 r3 r4 r* r �
wait_indexr| r} r
r
r r9 � st �� �/��'� �
� � � ��zEtcdClient.readr: c C sv t |t�s t�d� d S | �||�}i }|�� D ] \}}d}t |t�r%d}| j|||d�||< || d u r8 d S q|S )N�$etcd.update: fields is not type dictFT)rI )r> r? r rz r@ r= rJ )r& rG rB �keysrC rD �is_dirr
r
r r% s
�zEtcdClient.updatec
C sZ z|r | � |||�W S | �|||�W S tjy, } z
t�d|� W Y d}~dS d}~ww )rN rq N)rO rP rc r{ r rz )r& r1 rL rH rI r} r
r
r rJ s ��zEtcdClient.writec
C s� z | j j|||dd�}W nH tjtjtfy* } z
t�d|� W Y d }~d S d }~w tyB } z
t�d|� W Y d }~d S d }~w t yT } zt�d|� � d }~ww t
|d�S )NF�rH ro rq �*etcd: Could not connect to etcd server: %sr� rL ) re rJ rc � EtcdNotFile�EtcdRootReadOnlyrx r rz r r� ry �r& r1 rL rH r| r} r
r
r rP s � � ��
zEtcdClient.write_filec
C s� |d ur
t �d|� z | jj|d |dd�}W nX tjy' t �d|� Y dS tjtjtfyD } z
t � d|� W Y d }~d S d }~w t
y\ } z
t � d|� W Y d }~d S d }~w t yn } zt � d|� � d }~ww t |d�S ) Nz.etcd: non-empty value passed for directory: %sTr� z"etcd: directory already exists: %srq r� r� ro )
r �infore rJ rc r� Z
EtcdNotDirr� rx rz r r� ry r� r
r
r rO 0 s* � � ��
zEtcdClient.write_directoryc C s� i }z| � |�}W n tjtfy i Y S tjy' t�d|� Y d S w |jD ] }|jdu rC|j |kr8q+|j � d�}i ||< q+|j
||j < q+||iS )NzIetcd: failed to perform 'ls' operation on path %s due to connection errorTr; ) r9 rc rw rx r{ r rz �childrenro r1 rL )r& rB rF r= �itemZdir_namer
r
r rS I s( ��
z
EtcdClient.lsc
K s� |� dd �}|d urtjj�dd� |}|r t�d|� z| jj||d�r(W dS W dS t j
t j t j t j
t jtfyN } z
t�d|� W Y d }~d S d }~w tyf } z
t�d |� W Y d }~d S d }~w tyx } zt�d
|� � d }~ww ) Nr ri r� rk )r TFrq r� r� )rr rs rt ru rv r r! re rU rc r� r� ZEtcdDirNotEmptyrw r{ rx rz r r� )r& r1 r2 r* r r} r
r
r rU b sB � � � � ��zEtcdClient.deletec
C s� i }z| � |�}W n& tjtfy Y d S tjy/ } z
t�d|� W Y d }~d S d }~ww |jD ]'}t|j ��
d�}|j du rS|j |krHq3| � |j �||d < q3|j
||d < q3|S )Nrq r; T���)r9 rc rw rx r{ r rz r� r" r1 �splitro rW rL )r& rB rF r= r} r� �compsr
r
r rW � s$ ��
zEtcdClient.treerX rY rZ r[ r\ r. )r r
r r, r5 r r9 r% rJ rP rO rS rU rW �
__classcell__r
r
rf r r] W s
;
�
O
"r] c s� e Zd ZdZ d(� fdd� Zdd� Zdd � Zd
d � Zd d
� Zdd� Z d)dd�Z
d*dd�Z d+dd�Z d,dd�Z
d-dd�Zd-d d �Zd d � Zd*d d!�Zd"d#� Zd$d%� Zd&d'� Z� ZS ).� EtcdClientV3z�
.. versionadded:: 3005
Since etcd3 has no concept of directories, this class leaves write_directory unimplemented.
NFc s� t std��t�d� t� j|fi |�� | j�dd�r td��|p(| j�dd�| _ |p2| j�dd�| _
|p<| j�d d�| _ |pF| j�d
d�| _ |pP| j�d d �| _
| j�d
d �| jd< tjd| j| jd�| j��| _d S )Nz(Don't have etcd3-py, need to install it.z.etcd_util has the libraries needed for etcd v3r^ Tz%Can't create v3 with a v2 requirementzetcd.encode_keysFzetcd.encode_valuesz
etcd.raw_keyszetcd.raw_valueszetcd.unicode_errors�surrogateescaper Zverifyr_ r
)� HAS_ETCD_V3r r ra rb r, r r r � encode_keys�
encode_values�raw_keys�
raw_values�unicode_errorsr$ rr �etcd3rd r r re )r& r' r� r� r� r� r� r* rf r
r r, � s
�"zEtcdClientV3.__init__c K �X |� d| j� | jrtjjj|fd| ji|��}|S | js*t|t �r*|j
d| jd�}|S �Nr� �raw�UTF-8)�encoding�errors) �
setdefaultr� r� rs rt �msgpack�loadsr� r> �bytes�decode�r& r1 � extra_kwargsr
r
r �_maybe_decode_key� � �z EtcdClientV3._maybe_decode_keyc K �. |� d| j� | jrtjjj|fi |��}|S �Nr� )r� r� r� rs rt r� �dumpsr� r
r
r �_maybe_encode_key� � z EtcdClientV3._maybe_encode_keyc K r� r� ) r� r� r� rs rt r� r� r� r> r� r� �r& rL r� r
r
r �_maybe_decode_value� r� z EtcdClientV3._maybe_decode_valuec K r� r� )r� r� r� rs rt r� r� r� r
r
r �_maybe_encode_value� r� z EtcdClientV3._maybe_encode_valuec
C sV z| � |j�|_| �|j�|_W |S ty* } z
t�d|� W Y d }~|S d }~ww )Nzetcd3 decoding error: %s)r� r1 r� rL �AttributeErrorr rz )r& �kvr} r
r
r �
_decode_kv� s ���zEtcdClientV3._decode_kvr c K s� |� dd �}|d urtjj�dd� |}|r t�d|� |d dddd�}| j||d||d �}|d urJ|j|d
< |j |d < t
|d d�|d
< d|d< |S d S )Nrh ri rj rk Fr rl Trp r1 rL Z mod_revisionrn rm ) rr rs rt ru rv r r! r9 r1 rL ry ) r& r1 r2 r3 r4 r* rh rF r| r
r
r r5 � s0 � �
�zEtcdClientV3.watchc C s0 |s| � |�}t|t�r|�� jS d S | �|�S r. )r9 r> �listrr rL rW r~ r
r
r r s
zEtcdClientV3.getc
K sR |� dd �}|� dd �}|d urtjj�dd� |}|d ur(tjj�dd� |}|r0t�d|� |swz+| jj| � |�|d�} t
| dd �}
|
d u rQt� d |� |
W S |
D ]} | � | �} qS|
W S t
yv } z
t� d
| � W Y d } ~ d S d } ~ ww z| jj| � |�||d �}
|
j|d �}| � |�W S t
y� } z
t� d
| � W Y d } ~ d S d } ~ ww )Nr r� ri r� r� rk ��prefix�kvsz&etcd3 read: No values found for key %szetcd3 read: %s)r1 r� r4 )r3 zetcd3 watch: %s)rr rs rt ru rv r r! re �ranger� ry rz r� r� ZWatcherZ
watch_once)r& r1 r2 r8 r3 r4 r* r r� r| r� r� r} ZwatcherZ watch_eventr
r
r r9 sX �� � ��� ��zEtcdClientV3.readr: c C sf t |t�s t�d� d S | �||�}i }|�� D ]\}}t |t�r(t�d|� q| �||�||< q|S )Nr� z4etcd3 has no concept of directories, skipping key %s)r> r? r rz r@ r= r! rJ )r& rG rB r� rC rD r
r
r r% > s
zEtcdClientV3.updatec C s` |r | j j|d�}|�� | j j| �|�| �|�|jd� n
| j �| �|�| �|�� | �|�S )N)rH )�lease)re ZLeaseZgrant�putr� r� �IDr )r& r1 rL rH r� r
r
r rP L s ��
zEtcdClientV3.write_filec C s t d��)Nz etcd3 does not have directories)r rQ r
r
r rO [ s z EtcdClientV3.write_directoryc C s� i }| � |�}|d u r
i S | jsdnd}| jr t|t�r |n|jd| jd�}|�� D ]$\}}|�|�s:|| | }n|| }t|t�rJi ||| < q*|||< q*||iS )Nr; � /r� )r� ) rW r� r> r� �encoder� r= �endswithr? )r& rB rF rW �sepr1 rL Zret_keyr
r
r rS ^ s&
���
zEtcdClientV3.lsc K sj |� dd �}|d urtjj�dd� |}|r t�d|� | jj| � |�|d�}t
|d�r3|j r1dS d S dS ) Nr ri r� rk r� �deletedTF) rr rs rt ru rv r r! re Z delete_ranger� �hasattrr� )r& r1 r2 r* r r| r
r
r rU y s �
zEtcdClientV3.deletec C s^ | j sdnd}|�|��|d�^}}|r)||vr i ||< | �|d ||| � dS |||< dS )z$
Helper for _expand
r; r� � r N)r� �lstripr� �_expand_recurse)r& r1 rL �destr� Zouter�innerr
r
r r� � s z EtcdClientV3._expand_recursec C s( i }|� � D ] \}}| �|||� q|S )a�
This does the opposite of EtcdBase._flatten
For example, it will convert...
{
"/key1": "value1",
"/key2/subkey1": "subvalue1",
"/key2/subkey2": "subvalue2",
}
to...
{
"key1": "value1",
"key2": {
"subkey1": "subvalue1",
"subkey2": "subvalue2",
}
}
)r= r� )r& r� r� r1 rL r
r
r �_expand� s zEtcdClientV3._expandc s| | j � dd�}|d u r
d S | jsdnd}t|�dkr0|d j� kr0|�� }|j�|�d |jiS � fdd �|D �}| �|�S )
NT)r2 r; r� r� r r� c s" i | ]
}|j t� �d � |j�qS r. )r1 �lenrL )�.0r� �rB r
r �
<dictcomp>� s " z%EtcdClientV3.tree.<locals>.<dictcomp>)r9 r� r� r1 rr r� rL r� )r& rB r= r� r� r� r
r� r rW � s
zEtcdClientV3.tree)NNFFNrX rY rZ r[ r. )r r
r r r, r� r� r� r� r� r5 r r9 r% rP rO rS rU r� r� rW r� r
r
rf r r� � s8 �
�
6
r� c C sN | � di �}|� di �}i }|�|� |�|� |�| � |r%|�|i �S |S )NZpillarZmaster)r r% rr )r' r( Z opts_pillarZ opts_masterZ opts_mergedr
r
r r � s
r c K sp t | |d�}|�dd�}|r'tjj�dd� t|fddi|��}t�d� |S t |fddi|��}t�d� |S ) zk
Client creation at the module level.
This is the way users are meant to instantiate a client
)r( r^ TZ Potassiumzsetcd API v2 has been deprecated. It will be removed in the Potassium release, and etcd API v3 will be the default.r z<etcd_util will be attempting to use etcd API v2: python-etcdz9etcd_util will be attempting to use etcd API v3: etcd3-py)
r r rs rt ru rv r] r ra r� )r' r( r* r Zuse_v2re r
r
r �get_conn� s �
�
r� c C s
| � |�S )z3
Module level find tree at the given path.
)rW )re rB r
r
r rW � s
rW r. ) r �loggingZsalt.utils.msgpackrs Zsalt.utils.versionsZsalt.exceptionsr rc Zurllib3.exceptionsr r r` � ImportErrorr� r� � getLoggerr r r r r r� r r r] r� r r� rW r
r
r
r �<module> sD N � �
X D
'