:jddlZddlZddlZddlZddlmZddlmZmZddl Z ddl m cm Z ddl mZddlmZddlmZddlmZmZmZmZmZddl mZmZdd lmZgd ZGd d ZGd deZGddeZ e gZ!GddeZ"GddeZ#GddeZ$GddeZ%dZ&GddeZ'GddeZ(GddeZ)Gd d!eZ*Gd"d#eZ+Gd$d%eZ,Gd&d'eZ-Gd(d)eZ.Gd*d+eZ/Gd,d-eZ0Gd.d/eZ1Gd0d1eZ2Gd2d3eZ3y)4N)Sequence)OptionalUnion)Tensor) constraints) Distribution)_sum_rightmost broadcast_all lazy_propertytril_matrix_to_vecvec_to_tril_matrix)padsoftplus)_Number) AbsTransformAffineTransform CatTransformComposeTransformCorrCholeskyTransformCumulativeDistributionTransform ExpTransformIndependentTransformLowerCholeskyTransformPositiveDefiniteTransformPowerTransformReshapeTransformSigmoidTransformSoftplusTransform TanhTransformSoftmaxTransformStackTransformStickBreakingTransform Transformidentity_transformceZdZUdZdZej ed<ej ed<ddeddffd Z d Z e defd Z e dd Z e defd Zdd ZdZdZdZdZdZdZdZdZdZdZxZS)r#a Abstract class for invertable transformations with computable log det jacobians. They are primarily used in :class:`torch.distributions.TransformedDistribution`. Caching is useful for transforms whose inverses are either expensive or numerically unstable. Note that care must be taken with memoized values since the autograd graph may be reversed. For example while the following works with or without caching:: y = t(x) t.log_abs_det_jacobian(x, y).backward() # x will receive gradients. However the following will error when caching due to dependency reversal:: y = t(x) z = t.inv(y) grad(z.sum(), [y]) # error because z is x Derived classes should implement one or both of :meth:`_call` or :meth:`_inverse`. Derived classes that set `bijective=True` should also implement :meth:`log_abs_det_jacobian`. Args: cache_size (int): Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported. Attributes: domain (:class:`~torch.distributions.constraints.Constraint`): The constraint representing valid inputs to this transform. codomain (:class:`~torch.distributions.constraints.Constraint`): The constraint representing valid outputs to this transform which are inputs to the inverse transform. bijective (bool): Whether this transform is bijective. A transform ``t`` is bijective iff ``t.inv(t(x)) == x`` and ``t(t.inv(y)) == y`` for every ``x`` in the domain and ``y`` in the codomain. Transforms that are not bijective should at least maintain the weaker pseudoinverse properties ``t(t.inv(t(x)) == t(x)`` and ``t.inv(t(t.inv(y))) == t.inv(y)``. sign (int or Tensor): For bijective univariate transforms, this should be +1 or -1 depending on whether transform is monotone increasing or decreasing. Fdomaincodomain cache_sizereturnNcz||_d|_|dk(rn|dk(rd|_n tdt|y)Nr)NNzcache_size must be 0 or 1) _cache_size_inv _cached_x_y ValueErrorsuper__init__)selfr( __class__s s/home/mcse/projects/srt_converter/srt-converter-venv/lib/python3.12/site-packages/torch/distributions/transforms.pyr1zTransform.__init__bsB%@D ?  1_)D 89 9 cD|jj}d|d<|S)Nr-)__dict__copy)r2states r4 __getstate__zTransform.__getstate__ms" ""$f  r5c|jj|jjk(r|jjStd)Nz:Please use either .domain.event_dim or .codomain.event_dim)r& event_dimr'r/r2s r4r<zTransform.event_dimrs: ;; DMM$;$; ;;;(( (UVVr5cd}|j|j}|%t|}tj||_|S)z{ Returns the inverse :class:`Transform` of this transform. This should satisfy ``t.inv.inv is t``. N)r-_InverseTransformweakrefref)r2invs r4rBz Transform.invxsB  99 ))+C ;#D)C C(DI r5ct)z Returns the sign of the determinant of the Jacobian, if applicable. In general this only makes sense for bijective transforms. NotImplementedErrorr=s r4signzTransform.signs "!r5c|j|k(r|St|jtjurt||St t|d)Nr(z.with_cache is not implemented)r,typer1r#rEr2r(s r4 with_cachezTransform.with_cachesU   z )K :  )"4"4 44:4 4!T$ZL0N"OPPr5c ||uSNr2others r4__eq__zTransform.__eq__s u}r5c&|j| SrM)rQrOs r4__ne__zTransform.__ne__s;;u%%%r5c|jdk(r|j|S|j\}}||ur|S|j|}||f|_|S)z2 Computes the transform `x => y`. r)r,_callr.)r2xx_oldy_oldys r4__call__zTransform.__call__sY   q ::a= '' u :L JJqMa4r5c|jdk(r|j|S|j\}}||ur|S|j|}||f|_|S)z1 Inverts the transform `y => x`. r)r,_inverser.)r2rYrWrXrVs r4 _inv_callzTransform._inv_calls[   q ==# #'' u :L MM! a4r5ct)zD Abstract method to compute forward transformation. rDr2rVs r4rUzTransform._call "!r5ct)zD Abstract method to compute inverse transformation. rDr2rYs r4r\zTransform._inverser`r5ct)zU Computes the log det jacobian `log |dy/dx|` given input and output. rDr2rVrYs r4log_abs_det_jacobianzTransform.log_abs_det_jacobianr`r5c4|jjdzS)Nz())r3__name__r=s r4__repr__zTransform.__repr__s~~&&--r5c|S)z{ Infers the shape of the forward computation, given the input shape. Defaults to preserving shape. rNr2shapes r4 forward_shapezTransform.forward_shape  r5c|S)z} Infers the shapes of the inverse computation, given the output shape. Defaults to preserving shape. rNrjs r4 inverse_shapezTransform.inverse_shapermr5r)r)r#r+)rg __module__ __qualname____doc__ bijectiver Constraint__annotations__intr1r:propertyr<rBrFrKrQrSrZr]rUr\rerhrlro __classcell__r3s@r4r#r#1s*XI  " ""$$$ 3 t  W3WW   "c""Q&  " " " .r5r#ceZdZdZdeddffd ZejddZejdd Z e de fd Z e de fd Ze defd Zdd ZdZdZdZdZdZdZxZS)r?z| Inverts a single :class:`Transform`. This class is private; please instead use the ``Transform.inv`` property. transformr)NcHt||j||_yNrH)r0r1r,r-)r2r}r3s r4r1z_InverseTransform.__init__s  I$9$9:( r5F is_discretecJ|jJ|jjSrM)r-r'r=s r4r&z_InverseTransform.domains"yy$$$yy!!!r5cJ|jJ|jjSrM)r-r&r=s r4r'z_InverseTransform.codomains"yy$$$yyr5cJ|jJ|jjSrM)r-rur=s r4ruz_InverseTransform.bijectives"yy$$$yy"""r5cJ|jJ|jjSrM)r-rFr=s r4rFz_InverseTransform.signs yy$$$yy~~r5c|jSrM)r-r=s r4rBz_InverseTransform.invs yyr5ch|jJ|jj|jSrM)r-rBrKrJs r4rKz_InverseTransform.with_caches-yy$$$xx"":.222r5crt|tsy|jJ|j|jk(SNF) isinstancer?r-rOs r4rQz_InverseTransform.__eq__s3%!23yy$$$yyEJJ&&r5c`|jjdt|jdS)N())r3rgreprr-r=s r4rhz_InverseTransform.__repr__s)..))*!DO+>r5cT|jJ|jj|SrM)r-r]r_s r4rZz_InverseTransform.__call__ s'yy$$$yy""1%%r5cX|jJ|jj|| SrM)r-rerds r4rez&_InverseTransform.log_abs_det_jacobian s,yy$$$ ..q!444r5c8|jj|SrM)r-rorjs r4rlz_InverseTransform.forward_shapeyy&&u--r5c8|jj|SrM)r-rlrjs r4roz_InverseTransform.inverse_shaperr5rq)rgrrrsrtr#r1rdependent_propertyr&r'ryboolrurxrFrBrKrQrhrZrerlrorzr{s@r4r?r?s ))))$[##6"7"$[##6 7 #4##cY3' ?&5..r5r?c eZdZdZddeededdffd ZdZe jd d Z e jd d Z e defd Ze defd ZedefdZddZdZdZdZdZdZxZS)rab Composes multiple transforms in a chain. The transforms being composed are responsible for caching. Args: parts (list of :class:`Transform`): A list of transforms to compose. cache_size (int): Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported. partsr(r)Nc~|r|Dcgc]}|j|}}t| |||_ycc}wr)rKr0r1r)r2rr(partr3s r4r1zComposeTransform.__init__#s? =BCTT__Z0CEC J/ Ds:cVt|tsy|j|jk(Sr)rrrrOs r4rQzComposeTransform.__eq__)s#%!12zzU[[((r5Frc|jstjS|jdj}|jdjj }t |jD]R}||jj |jj z z }t||jj }T||j k\sJ||j kDr#tj|||j z }|S)Nr) rrrealr&r'r<reversedmax independent)r2r&r<rs r4r&zComposeTransform.domain.szz## #A%%JJrN++55 TZZ( >D ..1H1HH HIIt{{'<'<=I >F,,,,, v'' ' ,,VYAQAQ5QRF r5c|jstjS|jdj}|jdjj }|jD]R}||jj |jj z z }t ||jj }T||j k\sJ||j kDr#tj|||j z }|S)Nrr)rrrr'r&r<rr)r2r'r<rs r4r'zComposeTransform.codomain=szz## #::b>**JJqM((22 JJ @D 004;;3H3HH HIIt}}'>'>?I @H..... x)) )"..xXEWEW9WXHr5c:td|jDS)Nc34K|]}|jywrMru).0ps r4 z-ComposeTransform.bijective..Ns311;;3)allrr=s r4ruzComposeTransform.bijectiveLs3 333r5cJd}|jD]}||jz}|SNr+)rrF)r2rFrs r4rFzComposeTransform.signPs, !A!&&=D ! r5c$d}|j|j}|jtt|jDcgc]}|jc}}t j ||_t j ||_|Scc}wrM)r-rrrrBr@rA)r2rBrs r4rBzComposeTransform.invWsn 99 ))+C ;"8DJJ3G#HaAEE#HIC C(DI{{4(CH $IsB cR|j|k(r|St|j|Sr)r,rrrJs r4rKzComposeTransform.with_cachebs&   z )K zBBr5c8|jD] }||} |SrM)r)r2rVrs r4rZzComposeTransform.__call__gs#JJ DQA r5c p|jstj|S|g}|jddD]}|j||d|j|g}|jj }t |j|dd|ddD]x\}}}|jt|j||||jj z ||jj |jj z z }ztjtj|S)Nrr+)rtorch zeros_likeappendr&r<zipr rer' functoolsreduceoperatoradd)r2rVrYxsrtermsr<s r4rez%ComposeTransform.log_abs_det_jacobianlszz##A& &SJJsO $D IId2b6l # $ ! KK)) djj"Sb'2ab6: IJD!Q LL--a3YAVAV5V  004;;3H3HH HI  I e44r5cJ|jD]}|j|}|SrM)rrlr2rkrs r4rlzComposeTransform.forward_shapes*JJ .D&&u-E . r5c\t|jD]}|j|}|SrM)rrrors r4rozComposeTransform.inverse_shapes/TZZ( .D&&u-E . r5c|jjdz}|dj|jDcgc]}|j c}z }|dz }|Scc}w)Nz( z, z ))r3rgjoinrrh)r2 fmt_stringrs r4rhzComposeTransform.__repr__sT^^,,y8 innDJJ%Gqajjl%GHH e &HsA rprq)rgrrrsrtlistr#rxr1rQrrr&r'r rrurFryrBrKrZrerlrorhrzr{s@r4rrsd9o3t ) $[##6 7 $[##6 7 4444c YC  5*  r5rc eZdZdZ ddedededdffd ZddZejd d Z ejd d Z e de fd Ze defdZdZdZdZdZdZdZxZS)ra Wrapper around another transform to treat ``reinterpreted_batch_ndims``-many extra of the right most dimensions as dependent. This has no effect on the forward or backward transforms, but does sum out ``reinterpreted_batch_ndims``-many of the rightmost dimensions in :meth:`log_abs_det_jacobian`. Args: base_transform (:class:`Transform`): A base transform. reinterpreted_batch_ndims (int): The number of extra rightmost dimensions to treat as dependent. base_transformreinterpreted_batch_ndimsr(r)Nc`t|||j||_||_yr)r0r1rKrr)r2rrr(r3s r4r1zIndependentTransform.__init__s0 J/,77 C)B&r5ch|j|k(r|St|j|j|Sr)r,rrrrJs r4rKzIndependentTransform.with_caches5   z )K#   !?!?J  r5Frcjtj|jj|jSrM)rrrr&rr=s r4r&zIndependentTransform.domains,&&    & &(F(F  r5cjtj|jj|jSrM)rrrr'rr=s r4r'zIndependentTransform.codomains,&&    ( ($*H*H  r5c.|jjSrM)rrur=s r4ruzIndependentTransform.bijectives"",,,r5c.|jjSrM)rrFr=s r4rFzIndependentTransform.signs""'''r5c|j|jjkr td|j |SNToo few dimensions on input)dimr&r<r/rr_s r4rUzIndependentTransform._calls7 557T[[** *:; ;""1%%r5c|j|jjkr td|jj |Sr)rr'r<r/rrBrbs r4r\zIndependentTransform._inverses= 557T]],, ,:; ;""&&q))r5cj|jj||}t||j}|SrM)rrer r)r2rVrYresults r4rez)IndependentTransform.log_abs_det_jacobians1$$99!Q?(F(FG r5cz|jjdt|jd|jdS)Nrz, r)r3rgrrrr=s r4rhzIndependentTransform.__repr__s:..))*!D1D1D,E+FbIgIgHhhijjr5c8|jj|SrM)rrlrjs r4rlz"IndependentTransform.forward_shape""0077r5c8|jj|SrM)rrorjs r4roz"IndependentTransform.inverse_shaperr5rprq)rgrrrsrtr#rxr1rKrrr&r'ryrrurFrUr\rerhrlrorzr{s@r4rrs " C!C$'C C  C $[##6 7 $[##6 7 -4--(c((& *  k88r5rc eZdZdZdZ ddej dej deddffd Ze jd Z e jd Z dd Z d Zd ZdZdZdZxZS)ra Unit Jacobian transform to reshape the rightmost part of a tensor. Note that ``in_shape`` and ``out_shape`` must have the same number of elements, just as for :meth:`torch.Tensor.reshape`. Arguments: in_shape (torch.Size): The input event shape. out_shape (torch.Size): The output event shape. cache_size (int): Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported. (Default 0.) Tin_shape out_shaper(r)Nctj||_tj||_|jj |jj k7r t dt ||y)Nz6in_shape, out_shape have different numbers of elementsrH)rSizerrnumelr/r0r1)r2rrr(r3s r4r1zReshapeTransform.__init__sc  8, I. ==   DNN$8$8$: :UV V J/r5cptjtjt|jSrM)rrrlenrr=s r4r&zReshapeTransform.domains$&&{'7'7T]]9KLLr5cptjtjt|jSrM)rrrrrr=s r4r'zReshapeTransform.codomains$&&{'7'7T^^9LMMr5ch|j|k(r|St|j|j|Sr)r,rrrrJs r4rKzReshapeTransform.with_caches,   z )K t~~*UUr5c|jd|jt|jz }|j ||j zSrM)rkrrrreshaper)r2rV batch_shapes r4rUzReshapeTransform._call s?gg<#dmm*< <= yyt~~566r5c|jd|jt|jz }|j ||j zSrM)rkrrrrr)r2rYrs r4r\zReshapeTransform._inverses?gg=#dnn*= => yyt}}455r5c|jd|jt|jz }|j |SrM)rkrrr new_zeros)r2rVrYrs r4rez%ReshapeTransform.log_abs_det_jacobians6gg<#dmm*< <= {{;''r5c t|t|jkr tdt|t|jz }||d|jk7rtd||dd|j|d||jzSNrzShape mismatch: expected z but got )rrr/rr2rkcuts r4rlzReshapeTransform.forward_shapes u:DMM* *:; ;%j3t}}-- ;$-- '+E#$K= $--Q Tc{T^^++r5c t|t|jkr tdt|t|jz }||d|jk7rtd||dd|j|d||jzSr)rrr/rrs r4rozReshapeTransform.inverse_shape s u:DNN+ +:; ;%j3t~~.. ;$.. (+E#$K= $..AQR Tc{T]]**r5rprq)rgrrrsrtrurrrxr1rrr&r'rKrUr\rerlrorzr{s@r4rrs I  0** 0:: 0 0  0##M$M##N$NV 76(,+r5rc`eZdZdZej ZejZdZ dZ dZ dZ dZ dZy) rz8 Transform via the mapping :math:`y = \exp(x)`. Tr+c"t|tSrM)rrrOs r4rQzExpTransform.__eq__5%..r5c"|jSrM)expr_s r4rUzExpTransform._call8 uuwr5c"|jSrMlogrbs r4r\zExpTransform._inverse;rr5c|SrMrNrds r4rez!ExpTransform.log_abs_det_jacobian>r5Nrgrrrsrtrrr&positiver'rurFrQrUr\rerNr5r4rr+s=  F##HI D/r5rceZdZdZej Zej ZdZdde de ddffd Z ddZ e de fd Zd Zd Zd Zd ZdZdZxZS)rzD Transform via the mapping :math:`y = x^{\text{exponent}}`. Texponentr(r)NcJt||t|\|_yr)r0r1r r)r2rr(r3s r4r1zPowerTransform.__init__Ks" J/(2r5cR|j|k(r|St|j|Sr)r,rrrJs r4rKzPowerTransform.with_cacheOs&   z )Kdmm CCr5c6|jjSrM)rrFr=s r4rFzPowerTransform.signTs}}!!##r5ct|tsy|jj|jj j Sr)rrreqritemrOs r4rQzPowerTransform.__eq__Xs:%0}}/335::<|jd|jz Srrrbs r4r\zPowerTransform._inverse`suuQ&''r5c^|j|z|z jjSrM)rabsrrds r4rez#PowerTransform.log_abs_det_jacobiancs( !A%**,0022r5cXtj|t|jddSNrkrNrbroadcast_shapesgetattrrrjs r4rlzPowerTransform.forward_shapef"%%eWT]]GR-PQQr5cXtj|t|jddSrrrjs r4rozPowerTransform.inverse_shapeir r5rprq)rgrrrsrtrrr&r'rurrxr1rKr rFrQrUr\rerlrorzr{s@r4rrBs ! !F##HI33S33D $c$$= $(3RRr5rctj|j}tjtj||j d|j z SN?minr)rfinfodtypeclampsigmoidtinyeps)rVrs r4_clipped_sigmoidrms< KK E ;;u}}Q'UZZS599_ MMr5c`eZdZdZej ZejZdZ dZ dZ dZ dZ dZy) rzg Transform via the mapping :math:`y = \frac{1}{1 + \exp(-x)}` and :math:`x = \text{logit}(y)`. Tr+c"t|tSrM)rrrOs r4rQzSigmoidTransform.__eq__|%!122r5ct|SrM)rr_s r4rUzSigmoidTransform._calls ""r5ctj|j}|j|jd|j z }|j | jz Sr )rrrrrrrlog1p)r2rYrs r4r\zSigmoidTransform._inversesK AGG$ GG eiiG 8uuw1"%%r5c\tj|  tj|z SrM)Frrds r4rez%SigmoidTransform.log_abs_det_jacobians! A2A..r5N)rgrrrsrtrrr& unit_intervalr'rurFrQrUr\rerNr5r4rrrs=  F((HI D3#& /r5rc`eZdZdZej ZejZdZ dZ dZ dZ dZ dZy) rz Transform via the mapping :math:`\text{Softplus}(x) = \log(1 + \exp(x))`. The implementation reverts to the linear function when :math:`x > 20`. Tr+c"t|tSrM)rrrOs r4rQzSoftplusTransform.__eq__s%!233r5ct|SrMrr_s r4rUzSoftplusTransform._calls {r5cb| jjj|zSrM)expm1negrrbs r4r\zSoftplusTransform._inverses'zz|!%%'!++r5ct|  SrMr$rds r4rez&SoftplusTransform.log_abs_det_jacobians! }r5NrrNr5r4rrs=   F##HI D4,r5rcneZdZdZej ZejddZdZ dZ dZ dZ dZ d Zy ) ra Transform via the mapping :math:`y = \tanh(x)`. It is equivalent to .. code-block:: python ComposeTransform( [ AffineTransform(0.0, 2.0), SigmoidTransform(), AffineTransform(-1.0, 2.0), ] ) However this might not be numerically stable, thus it is recommended to use `TanhTransform` instead. Note that one should use `cache_size=1` when it comes to `NaN/Inf` values. grTr+c"t|tSrM)rrrOs r4rQzTanhTransform.__eq__s%//r5c"|jSrM)tanhr_s r4rUzTanhTransform._calls vvxr5c,tj|SrM)ratanhrbs r4r\zTanhTransform._inverses{{1~r5cVdtjd|z td|zz zS)N@g)mathrrrds r4rez"TanhTransform.log_abs_det_jacobians*dhhsma'(4!8*<<==r5N)rgrrrsrtrrr&intervalr'rurFrQrUr\rerNr5r4rrsF,  F#{##D#.HI D0 >r5rcReZdZdZej ZejZdZ dZ dZ y)rz*Transform via the mapping :math:`y = |x|`.c"t|tSrM)rrrOs r4rQzAbsTransform.__eq__rr5c"|jSrM)rr_s r4rUzAbsTransform._callrr5c|SrMrNrbs r4r\zAbsTransform._inverserr5N) rgrrrsrtrrr&rr'rQrUr\rNr5r4rrs*5   F##H/r5rc eZdZdZdZ ddeeefdeeefdededdf fd Z e defd Z e jd d Ze jd dZddZdZe deeeffdZdZdZdZdZdZxZS)ra Transform via the pointwise affine mapping :math:`y = \text{loc} + \text{scale} \times x`. Args: loc (Tensor or float): Location parameter. scale (Tensor or float): Scale parameter. event_dim (int): Optional size of `event_shape`. This should be zero for univariate random variables, 1 for distributions over vectors, 2 for distributions over matrices, etc. Tlocscaler<r(r)NcPt||||_||_||_yr)r0r1r8r9 _event_dim)r2r8r9r<r(r3s r4r1zAffineTransform.__init__s* J/ #r5c|jSrM)r;r=s r4r<zAffineTransform.event_dims r5Frc|jdk(rtjStjtj|jSNrr<rrrr=s r4r&zAffineTransform.domain7 >>Q ## #&&{'7'7HHr5c|jdk(rtjStjtj|jSr>r?r=s r4r'zAffineTransform.codomainr@r5c~|j|k(r|St|j|j|j|Sr)r,rr8r9r<rJs r4rKzAffineTransform.with_cache s7   z )K HHdjj$..Z  r5c8t|tsyt|jtr4t|jtr|j|jk7r7y|j|jk(j j syt|j tr5t|j tr|j |j k7ryy|j |j k(j j syy)NFT)rrr8rrrr9rOs r4rQzAffineTransform.__eq__s%1 dhh (Z 7-Kxx599$HH )..0557 djj' *z%++w/OzzU[[( JJ%++-22499;r5ct|jtr6t|jdkDrdSt|jdkrdSdS|jj S)Nrr+r)rr9rfloatrFr=s r4rFzAffineTransform.sign%sR djj' *djj)A-1 Utzz9JQ9N2 UTU Uzz  r5c:|j|j|zzSrMr8r9r_s r4rUzAffineTransform._call+sxx$**q.((r5c:||jz |jz SrMrGrbs r4r\zAffineTransform._inverse.sDHH  **r5c|j}|j}t|tr3t j |t jt|}n#t j|j}|jrQ|jd|j dz}|j|jd}|d|j }|j|S)N)rr)rkr9rrr full_liker1rrr<sizeviewsumexpand)r2rVrYrkr9r result_sizes r4rez$AffineTransform.log_abs_det_jacobian1s  eW %__QU(<=FYYu%))+F >> ++-(94>>/:UBK[[-11"5F+T^^O,E}}U##r5c tj|t|jddt|jddSrrrr r8r9rjs r4rlzAffineTransform.forward_shape>7%% 7488Wb174::wPR3S  r5c tj|t|jddt|jddSrrQrjs r4rozAffineTransform.inverse_shapeCrRr5rrrq)rgrrrsrtrurrrErxr1ryr<rrr&r'rKrQrFrUr\rerlrorzr{s@r4rrs I  $ 65= ! $VU]# $ $  $  $3$[##6I7I $[##6I7I  (!eFCK(!! )+ $  r5rcdeZdZdZej ZejZdZ dZ dZ d dZ dZ dZy) ra Transforms an uncontrained real vector :math:`x` with length :math:`D*(D-1)/2` into the Cholesky factor of a D-dimension correlation matrix. This Cholesky factor is a lower triangular matrix with positive diagonals and unit Euclidean norm for each row. The transform is processed as follows: 1. First we convert x into a lower triangular matrix in row order. 2. For each row :math:`X_i` of the lower triangular part, we apply a *signed* version of class :class:`StickBreakingTransform` to transform :math:`X_i` into a unit Euclidean length vector using the following steps: - Scales into the interval :math:`(-1, 1)` domain: :math:`r_i = \tanh(X_i)`. - Transforms into an unsigned domain: :math:`z_i = r_i^2`. - Applies :math:`s_i = StickBreakingTransform(z_i)`. - Transforms back into signed domain: :math:`y_i = sign(r_i) * \sqrt{s_i}`. Tctj|}tj|jj}|j d|zd|z }t |d}|dz}d|z jjd}|tj|jd|j|jz}|t|dddfddgd z}|S) Nrr+rdiag)rdevice.rvalue) rr,rrrrr sqrtcumprodeyerkrZr)r2rVrrzz1m_cumprod_sqrtrYs r4rUzCorrCholeskyTransform._call^s JJqMkk!''"&& GGSa#gG . qr * qDE<<>11"5  !''"+QWWQXXF F $S#2#X.Aa@ @r5c,dtj||zdz }t|dddfddgd}t|d}t|d}||j z }|j |j j z dz }|S) Nr+rr.rr[rWrY)rcumsumrr r]rr')r2rYy_cumsumy_cumsum_shiftedy_vec y_cumsum_vectrVs r4r\zCorrCholeskyTransform._inversemsu||AEr22xSbS1Aq6C"12.)*:D \'') ) WWY (A -r5Ncd||zjdz }t|d}d|jjdz}d|t d|zzt jdz jdz}||zS)Nr+rrdrW?r0)rer rrMrr1)r2rVrY intermediates y1m_cumsumy1m_cumsum_trilstick_breaking_logdet tanh_logdets r4rez*CorrCholeskyTransform.log_abs_det_jacobianys !a%B// -ZbA #&;&;&=&A&A"&E EAa 00488C=@EE"EMM ${22r5ct|dkr td|d}tdd|zzdzdz}||dz zdz|k7r td|dd||fzS)Nr+rrg?rYrmz-Input is not a flattend lower-diagonal number)rr/round)r2rkNDs r4rlz#CorrCholeskyTransform.forward_shapest u:>:; ; "I 4!a%:; ; 9b !23 3 "I QK1 SbzQD  r5rM)rgrrrsrtr real_vectorr& corr_choleskyr'rurUr\rerlrorNr5r4rrIs=  $ $F((HI   3#!r5rc^eZdZdZej ZejZdZ dZ dZ dZ dZ y)r a< Transform from unconstrained space to the simplex via :math:`y = \exp(x)` then normalizing. This is not bijective and cannot be used for HMC. However this acts mostly coordinate-wise (except for the final normalization), and thus is appropriate for coordinate-wise optimization algorithms. c"t|tSrM)rr rOs r4rQzSoftmaxTransform.__eq__rr5c||}||jdddz j}||jddz S)NrTr)rrrM)r2rVlogprobsprobss r4rUzSoftmaxTransform._calls@HLLT2155::<uyyT***r5c&|}|jSrMr)r2rYrs r4r\zSoftmaxTransform._inversesyy{r5c8t|dkr td|SNr+rrxrjs r4rlzSoftmaxTransform.forward_shape u:>:; ; r5c8t|dkr td|Srrxrjs r4rozSoftmaxTransform.inverse_shaperr5N)rgrrrsrtrryr&simplexr'rQrUr\rlrorNr5r4r r s8 $ $F""H3+  r5r cheZdZdZej ZejZdZ dZ dZ dZ dZ dZdZy ) r"a Transform from unconstrained space to the simplex of one additional dimension via a stick-breaking process. This transform arises as an iterated sigmoid transform in a stick-breaking construction of the `Dirichlet` distribution: the first logit is transformed via sigmoid to the first probability and the probability of everything else, and then the process recurses. This is bijective and appropriate for use in HMC; however it mixes coordinates together and is less appropriate for optimization. Tc"t|tSrM)rr"rOs r4rQzStickBreakingTransform.__eq__%!788r5c(|jddz|j|jdjdz }t||j z }d|z j d}t |ddgdt |ddgdz}|S)Nrr+rr[)rknew_onesrerrr^r)r2rVoffsetra z_cumprodrYs r4rUzStickBreakingTransform._callsq1::aggbk#:#A#A"#EE Q- .UOOB' Aq6 #c)aV1&E Er5c|dddf}|jd|j|jdjdz }d|jdz }tj|tj |j j}|j|jz |jz}|S)N.rr+)r) rkrrerrrrrr)r2rYy_croprsfrVs r4r\zStickBreakingTransform._inverses38qzz&,,r*:;BB2FF r" "[[QWW!5!:!: ; JJL2668 #fjjl 2r5c,|jddz|j|jdjdz }||jz }| t j |z|dddfjzj d}|S)Nrr+.)rkrrerr logsigmoidrM)r2rVrYrdetJs r4rez+StickBreakingTransform.log_abs_det_jacobiansq1::aggbk#:#A#A"#EE  Q\\!_$qcrc{'88==bA r5cRt|dkr td|dd|ddzfzSNr+rrrxrjs r4rlz$StickBreakingTransform.forward_shape5 u:>:; ;SbzU2Y],,,r5cRt|dkr td|dd|ddz fzSrrxrjs r4roz$StickBreakingTransform.inverse_shaperr5N)rgrrrsrtrryr&rr'rurQrUr\rerlrorNr5r4r"r"sB  $ $F""HI9- -r5r"cteZdZdZej ej dZejZ dZ dZ dZ y)rz Transform from unconstrained matrices to lower-triangular matrices with nonnegative diagonal entries. This is useful for parameterizing positive definite matrices in terms of their Cholesky factorization. rYc"t|tSrM)rrrOs r4rQzLowerCholeskyTransform.__eq__rr5c|jd|jddjjzSNrrl)dim1dim2)trildiagonalr diag_embedr_s r4rUzLowerCholeskyTransform._call4vvbzAJJBRJ8<<>IIKKKr5c|jd|jddjjzSr)rrrrrbs r4r\zLowerCholeskyTransform._inverse rr5N) rgrrrsrtrrrr&lower_choleskyr'rQrUr\rNr5r4rrs?%[ $ $[%5%5q 9F))H9LLr5rcteZdZdZej ej dZejZ dZ dZ dZ y)rzN Transform from unconstrained matrices to positive-definite matrices. rYc"t|tSrM)rrrOs r4rQz PositiveDefiniteTransform.__eq__s%!:;;r5c@t|}||jzSrM)rmTr_s r4rUzPositiveDefiniteTransform._calls $ " $Q '144xr5crtjj|}tj |SrM)rlinalgcholeskyrrBrbs r4r\z"PositiveDefiniteTransform._inverses* LL ! !! $%'++A..r5N) rgrrrsrtrrrr&positive_definiter'rQrUr\rNr5r4rrs=%[ $ $[%5%5q 9F,,H</r5rc eZdZUdZeeed< ddeedede eededdf fd Z e defd Z e defd Z dd Zd ZdZdZedefdZej,dZej,dZxZS)ra Transform functor that applies a sequence of transforms `tseq` component-wise to each submatrix at `dim`, of length `lengths[dim]`, in a way compatible with :func:`torch.cat`. Example:: x0 = torch.cat([torch.range(1, 10), torch.range(1, 10)], dim=0) x = torch.cat([x0, x0], dim=0) t0 = CatTransform([ExpTransform(), identity_transform], dim=0, lengths=[10, 10]) t = CatTransform([t0, t0], dim=0, lengths=[20, 20]) y = t(x) transformsNtseqrlengthsr(r)cvtd|DsJ|r|Dcgc]}|j|}}t| |t ||_|dgt |j z}t ||_t |jt |j k(sJ||_ycc}w)Nc3<K|]}t|tywrMrr#rrjs r4rz(CatTransform.__init__..:::a+:rHr+) rrKr0r1rrrrr)r2rrrr(rjr3s r4r1zCatTransform.__init__3s:T:::: 6:;ALL,;D; J/t* ?cC00GG} 4<< C$8888.G811;;8r)rrr=s r4r<zCatTransform.event_dimE8888r5c,t|jSrM)rMrr=s r4lengthzCatTransform.lengthIs4<<  r5c||j|k(r|St|j|j|j|SrM)r,rrrrrJs r4rKzCatTransform.with_cacheMs2   z )KDOOTXXt||ZPPr5c|j |jcxkr|jksJJ|j|j|jk(sJg}d}t|j|j D]>\}}|j |j||}|j||||z}@tj||jSNrrd) rrKrrrrnarrowrrcat)r2rVyslicesstarttransrxslices r4rUzCatTransform._callRsx488-aeeg-----vvdhh4;;... $,,? #ME6XXdhhv6F NN5= )FNE #yydhh//r5c|j |jcxkr|jksJJ|j|j|jk(sJg}d}t|j|j D]G\}}|j |j||}|j|j|||z}Itj||jSr) rrKrrrrrrrBrr)r2rYxslicesrrryslices r4r\zCatTransform._inverse]sx488-aeeg-----vvdhh4;;... $,,? #ME6XXdhhv6F NN599V, -FNE #yydhh//r5c|j |jcxkr|jksJJ|j|j|jk(sJ|j |jcxkr|jksJJ|j|j|jk(sJg}d}t|j|j D]\}}|j |j||}|j |j||}|j||} |j|jkr#t| |j|jz } |j| ||z}|j} | dk\r| |jz } | |jz} | dkrtj|| St|Sr)rrKrrrrrrer<r rrrrM) r2rVrY logdetjacsrrrrr logdetjacrs r4rez!CatTransform.log_abs_det_jacobianhsx488-aeeg-----vvdhh4;;...x488-aeeg-----vvdhh4;;...  $,,? #ME6XXdhhv6FXXdhhv6F2266BI/*9dnnu6VW   i (FNE #hh !8-CDNN" 799ZS1 1z? "r5c:td|jDS)Nc34K|]}|jywrMrrs r4rz)CatTransform.bijective..rrrrr=s r4ruzCatTransform.bijectiverr5ctj|jDcgc]}|jc}|j|j Scc}wrM)rrrr&rrr2rjs r4r&zCatTransform.domains8# /!QXX /4<<  /Actj|jDcgc]}|jc}|j|j Scc}wrM)rrrr'rrrs r4r'zCatTransform.codomains8!% 1AQZZ 1488T\\  1r)rNrrq)rgrrrsrtrr#rwrrxrr1r r<rrKrUr\reryrrurrr&r'rzr{s@r4rr"s Y +/ y!(3-(     $9399!!!Q 0 0#29499## $ ## $ r5rc eZdZUdZeeed< ddeedededdffd Z dd Z d Z d Z d Z d ZedefdZej&dZej&dZxZS)r!aW Transform functor that applies a sequence of transforms `tseq` component-wise to each submatrix at `dim` in a way compatible with :func:`torch.stack`. Example:: x = torch.stack([torch.range(1, 10), torch.range(1, 10)], dim=1) t = StackTransform([ExpTransform(), identity_transform], dim=1) y = t(x) rrrr(r)Nctd|DsJ|r|Dcgc]}|j|}}t| |t ||_||_ycc}w)Nc3<K|]}t|tywrMrrs r4rz*StackTransform.__init__..rrrH)rrKr0r1rrr)r2rrr(rjr3s r4r1zStackTransform.__init__s_:T:::: 6:;ALL,;D; J/t*.rrrr=s r4ruzStackTransform.bijectiverr5ctj|jDcgc]}|jc}|jScc}wrM)rrrr&rrs r4r&zStackTransform.domains/  DOO!Dq!((!DdhhOO!DActj|jDcgc]}|jc}|jScc}wrM)rrrr'rrs r4r'zStackTransform.codomains/  doo!F!**!FQQ!FrrTrq)rgrrrsrtrr#rwrrxr1rKrrUr\reryrrurrr&r'rzr{s@r4r!r!s YJKY'.1CF E H22 59499##P$P##R$Rr5r!ceZdZdZdZej ZdZdde de ddffd Z e de ejfd Zd Zd Zd Zdd ZxZS)raA Transform via the cumulative distribution function of a probability distribution. Args: distribution (Distribution): Distribution whose cumulative distribution function to use for the transformation. Example:: # Construct a Gaussian copula from a multivariate normal. base_dist = MultivariateNormal( loc=torch.zeros(2), scale_tril=LKJCholesky(2).sample(), ) transform = CumulativeDistributionTransform(Normal(0, 1)) copula = TransformedDistribution(base_dist, [transform]) Tr+ distributionr(r)Nc4t||||_yr)r0r1r)r2rr(r3s r4r1z(CumulativeDistributionTransform.__init__s J/(r5c.|jjSrM)rsupportr=s r4r&z&CumulativeDistributionTransform.domains  (((r5c8|jj|SrM)rcdfr_s r4rUz%CumulativeDistributionTransform._calls  $$Q''r5c8|jj|SrM)ricdfrbs r4r\z(CumulativeDistributionTransform._inverses  %%a((r5c8|jj|SrM)rlog_probrds r4rez4CumulativeDistributionTransform.log_abs_det_jacobians  ))!,,r5cR|j|k(r|St|j|Sr)r,rrrJs r4rKz*CumulativeDistributionTransform.with_caches(   z )K.t/@/@ZXXr5rprq)rgrrrsrtrurr r'rFrrxr1ryrrvr&rUr\rerKrzr{s@r4rrst$I((H D)\)s)4))!7!78))()-Yr5r)4rr1rr@collections.abcrtypingrrrtorch.nn.functionalnn functionalrrtorch.distributionsr torch.distributions.distributionrtorch.distributions.utilsr r r r r rr torch.typesr__all__r#r?rr$rrrrrrrrrrrr r"rrrr!rrNr5r4rsh $" +9. 0ffR;. ;.|wywt&b)I89I8XG+yG+T9.(RY(RVN /y/2 0*>I*>Z 9  f if RP!IP!f!y!H5-Y5-pLYL,/ /(m 9m `GRYGRT+Yi+Yr5