GJj6 ddlZddlmZmZddlmZejfdedejdeejgejffdZ ddejfd ed edejdeejgejffd Z ddejfd ed edejdeejgejffdZ ejfdejdeejgejffdZdZejfdejdeejegejffdZejfdejdeejegejffdZejfdejdeejedegejffdZejfdejdeejedegejffdZddejfded ed edejdeejgejff dZdejfdedejdeejgejffdZy)N)CallableLiteralvaluedtypereturncXdtjdtjffd }|S)aAn initializer that returns an array filled with ``value``. Args: value (float): The value to fill the array with. dtype (Dtype, optional): The data type of the array. Default: ``float32``. Returns: Callable[[array], array]: An initializer that returns an array with the same shape as the input, filled with ``value``. Example: >>> init_fn = nn.init.constant(0.5) >>> init_fn(mx.zeros((2, 2))) array([[0.5, 0.5], [0.5, 0.5]], dtype=float32) arcHtj|jSNr)mxfullshape)r rrs U/Users/ahmed/devFolder/claude-voice/.venv/lib/python3.12/site-packages/mlx/nn/init.py initializerzconstant..initializerswwqwwU33r array)rrrs`` rconstantr s&,4rxx4BHH4 rg?meanstdc\dtjdtjffd }|S)aAn initializer that returns samples from a normal distribution. Args: mean (float, optional): Mean of the normal distribution. Default: ``0.0``. std (float, optional): Standard deviation of the normal distribution. Default: ``1.0``. dtype (Dtype, optional): The data type of the array. Default: ``float32``. Returns: Callable[[array], array]: An initializer that returns an array with the same shape as the input, filled with samples from a normal distribution. Example: >>> init_fn = nn.init.normal() >>> init_fn(mx.zeros((2, 2))) array([[-0.982273, -0.534422], [0.380709, 0.0645099]], dtype=float32) r rc^tjj|jS)Nrscalelocr)r randomnormalr)r rrrs rrznormal..initializer>s%yyaggSd%PPrr)rrrrs``` rrr%s)2QrxxQBHHQ rlowhighc\dtjdtjffd }|S)aAn initializer that returns samples from a uniform distribution. Args: low (float, optional): The lower bound of the uniform distribution. Default: ``0.0``. high (float, optional): The upper bound of the uniform distribution. Default: ``1.0`` dtype (Dtype, optional): The data type of the array. Default: ``float32``. Returns: Callable[[array], array]: An initializer that returns an array with the same shape as the input, filled with samples from a uniform distribution Example: >>> init_fn = nn.init.uniform(low=0, high=1) >>> init_fn(mx.zeros((2, 2))) array([[0.883935, 0.863726], [0.617261, 0.417497]], dtype=float32) r rc^tjj|jSr )r runiformr)r rr!r s rrzuniform..initializer]s%yy  dAGG5 AArr)r r!rrs``` rr$r$Ds)2BrxxBBHHB rcTdtjdtjffd }|S)aAn initializer that returns an identity matrix. Args: dtype (Dtype, optional): The data type of the array. Default: ``float32``. Returns: Callable[[array], array]: An initializer that returns an identity matrix with the same shape as the input. Example: >>> init_fn = nn.init.identity() >>> init_fn(mx.zeros((2, 2))) array([[1, 0], [0, 1]], dtype=float32) arrrc|jdk7s|jd|jdk7rtd|jdtj|jdS)Nrz6The input array must be a square matrix but got shape .)nr)ndimr ValueErrorr eye)r&rs rrzidentity..initializervs^ 88q=CIIaLCIIaL8H STU vv ! E22rrrrs` ridentityr0cs&&33bhh3 rc|jdkrtd|jd|jd}|jd}|jdkDr%d}|jddD]}||z} ||z}||z}||fS)Nr(zPGlorot / He initialization requires at least 2 dimensional input but input with z dimensions.rr))r,r-r)xfan_infan_outreceptive_fieldds r_calculate_fan_in_fan_outr8svvz  vvhl 4  WWR[FggajGvvz2A q O/)O+ 7?rcbddtjdtdtjffd }|S)aWA Glorot normal initializer. This initializer samples from a normal distribution with a standard deviation computed from the number of input (``fan_in``) and output (``fan_out``) units according to: .. math:: \sigma = \gamma \sqrt{\frac{2.0}{\text{fan\_in} + \text{fan\_out}}} For more details see the original reference: `Understanding the difficulty of training deep feedforward neural networks `_ Args: dtype (Dtype, optional): The data type of the array. Default: ``float32``. Returns: Callable[[array, float], array]: An initializer that returns an array with the same shape as the input, filled with samples from the Glorot normal distribution. Example: >>> init_fn = nn.init.glorot_normal() >>> init_fn(mx.zeros((2, 2))) array([[0.191107, 1.61278], [-0.150594, -0.363207]], dtype=float32) >>> init_fn(mx.zeros((2, 2)), gain=4.0) array([[1.89613, -4.53947], [4.48095, 0.995016]], dtype=float32) r gainrct|\}}|tjd||zz z}tjj |j |S)Ng@rrr)r8mathsqrtr rrr)r r:r4r5rrs rrz"glorot_normal..initializersN3A6TYYsfw&6788yyaggSFFrrr rfloatr/s` r glorot_normalrBs2FGrxxGuGrxxG rcbddtjdtdtjffd }|S)aQA Glorot uniform initializer. This initializer samples from a uniform distribution with a range computed from the number of input (``fan_in``) and output (``fan_out``) units according to: .. math:: \sigma = \gamma \sqrt{\frac{6.0}{\text{fan\_in} + \text{fan\_out}}} For more details see the original reference: `Understanding the difficulty of training deep feedforward neural networks `_ Args: dtype (Dtype, optional): The data type of the array. Default: ``float32``. Returns: Callable[[array, float], array]: An initializer that returns an array with the same shape as the input, filled with samples from the Glorot uniform distribution. Example: >>> init_fn = nn.init.glorot_uniform() >>> init_fn(mx.zeros((2, 2))) array([[0.223404, -0.890597], [-0.379159, -0.776856]], dtype=float32) >>> init_fn(mx.zeros((2, 2)), gain=4.0) array([[-1.90041, 3.02264], [-0.912766, 4.12451]], dtype=float32) r r:rct|\}}|tjd||zz z}tjj | ||j S)Ng@r )r8r=r>r rr$r)r r:r4r5limitrs rrz#glorot_uniform..initializersR3A6tyy(8!9::yy  %u EErr?r@r/s` rglorot_uniformrFs2FFrxxFuFrxxF rr4r5c x ddtjdtddtdtjffd }|S)aBuild a He normal initializer. This initializer samples from a normal distribution with a standard deviation computed from the number of input (``fan_in``) or output (``fan_out``) units according to: .. math:: \sigma = \gamma \frac{1}{\sqrt{\text{fan}}} where :math:`\text{fan}` is either the number of input units when the ``mode`` is ``"fan_in"`` or output units when the ``mode`` is ``"fan_out"``. For more details see the original reference: `Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification `_ Args: dtype (Dtype, optional): The data type of the array. Default: ``float32``. Returns: Callable[[array, str, float], array]: An initializer that returns an array with the same shape as the input, filled with samples from the He normal distribution. Example: >>> init_fn = nn.init.he_normal() >>> init_fn(mx.zeros((2, 2))) # uses fan_in array([[-1.25211, 0.458835], [-0.177208, -0.0137595]], dtype=float32) >>> init_fn(mx.zeros((2, 2)), mode="fan_out", gain=5) array([[5.6967, 4.02765], [-4.15268, -2.75787]], dtype=float32) r moderGr:rct|\}}|dk(r|}n|dk(r|}ntd|d|tj|z }tj j |j|S)Nr4r5Invalid mode: ". Valid modes are: fan_in, fan_outr<)r8r-r=r>r rrr)r rIr:r4r5fanrrs rrzhe_normal..initializerst 4A6 8 C Y C~dV3UVW WTYYs^#yyaggSFFrr4rr rrrAr/s` r he_normalrPsPR.6G 88G)*GG  G rc x ddtjdtddtdtjffd }|S)aA He uniform (Kaiming uniform) initializer. This initializer samples from a uniform distribution with a range computed from the number of input (``fan_in``) or output (``fan_out``) units according to: .. math:: \sigma = \gamma \sqrt{\frac{3.0}{\text{fan}}} where :math:`\text{fan}` is either the number of input units when the ``mode`` is ``"fan_in"`` or output units when the ``mode`` is ``"fan_out"``. For more details see the original reference: `Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification `_ Args: dtype (Dtype, optional): The data type of the array. Default: ``float32``. Returns: Callable[[array, str, float], array]: An initializer that returns an array with the same shape as the input, filled with samples from the He uniform distribution. Example: >>> init_fn = nn.init.he_uniform() >>> init_fn(mx.zeros((2, 2))) # uses fan_in array([[0.0300242, -0.0184009], [0.793615, 0.666329]], dtype=float32) >>> init_fn(mx.zeros((2, 2)), mode="fan_out", gain=5) array([[-1.64331, -2.16506], [1.08619, 5.79854]], dtype=float32) r rIrGr:rct|\}}|dk(r|}n|dk(r|}ntd|d|tjd|z z}tj j | ||jS)Nr4r5rKrLg@r )r8r-r=r>r rr$r)r rIr:r4r5rMrErs rrzhe_uniform..initializerNs} 4A6 8 C Y C~dV3UVW Wtyys++yy  %u EErrNrOr/s` r he_uniformrS%sPV.6F 88F)*FF  F rsparsityc`dtjdtjffd }|S)aAn initializer that returns a sparse matrix. Args: sparsity (float): The fraction of elements in each column to be set to zero. mean (float, optional): Mean of the normal distribution. Default: ``0.0``. std (float, optional): Standard deviation of the normal distribution. Default: ``1.0``. dtype (Dtype, optional): The data type of the array. Default: ``float32``. Returns: Callable[[array], array]: An initializer that returns an array with the same shape as the input, filled with samples from a normal distribution. Example: >>> init_fn = nn.init.sparse(sparsity=0.5) >>> init_fn(mx.zeros((2, 2))) array([[-1.91187, -0.117483], [0, 0]], dtype=float32) r rc|jdk7r td|j\}}tt j |z}t jt jj|jd}t jj|j}d|t j|j|d|ddd|ff<|S)Nr(z,Only tensors with 2 dimensions are supportedrr))axisrr) r,r-rintr=ceilr argsortrr$rarangereshape) r rowscols num_zerosorderrrrTrs rrzsparse..initializers 66Q;KL LWW d (T/23  299,,177,;!D II  177#4u  MDE"))D/ ! !$ *E!ZiZ-,@ @Arr)rTrrrrs```` rsparserbas+< rxx BHH   rr:cXdtjdtjffd }|S)aAn initializer that returns an orthogonal matrix. Args: gain (float, optional): Scaling factor for the orthogonal matrix. Default: ``1.0``. dtype (Dtype, optional): Data type of the array. Default: ``float32``. Returns: Callable[[array], array]: An initializer that returns an orthogonal matrix with the same shape as the input. r rc|jdk7r td|j\}}t||}tj j ||f}tjj|tj\}}t j|}|t j|z}|d|d|f}| z}|jS)Nr(zHOrthogonal initialization requires a 2D array but got a {a.ndim}D array.rW)stream) r,r-rmaxr rrlinalgqrcpudiagsignastype) r r^r_r+rmatqrr7rr:s rrzorthogonal..initializers 66Q;&  WW d dOyyq!f-yy||D|01 GGAJ  N eteUdUlO Hxxrr)r:rrs`` r orthogonalrps&rxxBHH4 r)r=typingrrmlx.corecorer float32rADtyperrrr$r0r8rBrFrPrSrbrprrrws $%'JJ  rxxj"(("#:C2:: !02 rxxj"(("#@C2:: !02 rxxj"(("#> "zzBHHhz2887K.L:,jj( 88( rxx)*(Xjj( 88( rxx)*(Xjj7 887 rxx!45u=rxxGH7vjj9 889 rxx!45u=rxxGH9|jj ,, , , 88 , rxxj"(("# ,`) ) hh) rxxj"(("#)r