Statistical inference – A basic question about a randomized test involving Type I error.

I have a fundamental question in the context of the verification of statistical assumptions, specifically randomized tests. Suposse that I have two actions (alternatives) on a certain unknown parameter $ theta in Theta $: the null ($ H_0 $) and alternative hypotheses ($ H_1 $).

In this case, the sample space is $ (0,15) subset mathbb {R} $. We know that the critical function is given by
$$ phi (x) = P (reject , , H_0 , , | , , x , , observed) $$

I do not know exactly if this definition really implies a conditional probability. Suposse I have the following critical function

$$
phi (x) =
begin {cases}
0, quad x in (0,2) \
p, quad x in (2,10) \
1, quad x in (10,15) \
end {cases}
$$

I can understand why

$$ P (reject , , H_0 , , , , H_0 , , is , , true) , = 0 times P (x in (0,2)) + p times P (x in (2,10)) + 1 times P (x in (10,15)) $$

The right side looks a lot like a wait. But I can not understand.

Randomized algorithms – Generate a random matrix and its inverse

I want to randomly generate a pair of invertible matrices $ A, B $ which are inverses of each other. In other words, I want to sample randomly randomly in all pairs $ A, B $ of matrices such as $ AB = BA = text {Id} $.

Is there an effective way to do it? Can we do it with the expected running time approaching $ O (n ^ 2) $?

Suppose we work with $ n times n $ Boolean matrices (all inputs 0 or 1, arithmetic made modulo 2). I'm fine with an approximate algorithm (say that it samples a distribution exponentially close to the desired distribution). My motivation is only curiosity; I do not have any practical application in mind.


The obvious approach is to generate a random invertible matrix $ A $, calculate its inverse and rule $ A = B ^ {- 1} $. This has time $ O (n ^ omega) $, or $ omega $ is the multiplication constant of the matrix – something in the neighborhood of $ O (n ^ 3) $ in practice. Can we do better?

An approach that has come to my mind is to choose a whole $ T $ simple linear transformations on matrices such as, for each $ t in T $, we can apply the changes $ M mapsto tM $ and $ M mapsto Mt ^ {- 1} $ in $ O (1) $ time. Then we could define $ A_0 = B_0 = text {Id} $and at a pace $ i $, sample a chance $ t $ of $ T $, together $ A_ {i + 1} = tA_i $ and $ B_ {i + 1} = B_it ^ {- 1} $and repeat the operation for a number of steps (for example $ O (n ^ 2 log n) $ iterations). However, I do not know how we could prove how fast it approaches the desired distribution.

complexity theory – Is there a deterministic algorithm for a randomized algorithm for a problem?

If the runtime of the algorithm is limited, in that for each entry $ x $ there is a number $ T (x) $ so that whatever the coin throws, the algorithm always ends in $ T (x) $ Step by step, you can always "try all possibilities" and convert your randomized algorithm into a deterministic algorithm, albeit with a much longer runtime.

For polynomial time algorithms in particular, it is assumed that you can discard any algorithm with at most a polynomial swelling in time, conjecture usually defined as follows: $ mathsf {P} = mathsf {BPP} $.

For a randomized Turing machine, why are we throwing a coin?

What effect would it have if we said, we decided that our randomization would come from rolling a fair die?

Is it possible for the coin to simulate a more complex situation like a fair die roll?

Randomized algorithms – How can a maximum number of minimum cuts in a graph be exactly $ n choose $ 2?

According to my instructor, $ n choose $ 2 is the maximum number of minimum cuts that we can have on a graph. To prove it, he showed the lower limit using an n cycle chart. To prove the upper limit, he drew the argument of two facts:

  • Probability of finding $ i ^ {th} $ min cut $ geq frac {2} {n (n-1)} = frac {1} {n choose 2} $
  • Event to find $ i ^ {th} $ min cut is disjointed.

So, by adding the probabilities, he proved the upper limit of $ n choose $ 2.

Now, if we look at a tree, graphically, with $ n $ nodes, then we will be able to conclude $ (n-1) $ cuts min which is less than $ n choose $ 2 cuts ($ n geq3) $. Did I miss something?

sorting – Randomized Quick Select – depends on the time complexity of k – the index to select

I analyzed the time complexity of random quick selections, according to n – the size of the input and k – the index of the element to select.
The time dependence of n on temporal complexity was linear, as expected, but it was based on an inverted parabola – that is, the time complexity was greatest when you tried to select the median and the weakest when you tried to select the first or last element of the array.

Is there an explanation for this?

Randomized algorithms – Can we generate random numbers using irrational numbers such as π and e?

Irrational numbers like $ pi $, $ e $ and $ sqrt {2} $ have a unique and non-repeating sequence after the decimal point. If we extract the $ n $-the number of these figures (where $ n $ is the number of times the method is called) and make a number with the numbers as is, should not we get a perfect random number generator? For example, if we use $ sqrt {2} $, $ e $ and $ pi $the first number is 123, the second number 471, the next number 184, and so on.

python 3.x – Randomized random walk of N dimensions in Cartesian coordinates

I have written a random walk routine on which I hope to build in the future. Before that, I hoped to get some criticism.

I believe the implementation is correct. I noticed that many other implementations found online use of loops or modules with which I am unfamiliar.
My goal was to vectorize the walk in an n dimensional space with the optional use of boundary conditions.

The main idea is to generate an array with n-dimensional random numbers depending on the desired distribution. For the moment, only the "normal" distribution is implemented. If a threshold is not defined, the average of the distribution is used as the threshold. Numbers above this threshold are taken in the positive direction, while numbers below this threshold are taken in the negative direction. If the number is exactly equal to this threshold, no action is taken. The initial stages table (called based initially consists of all zeros; the indices corresponding to the positive and negative steps are used to hide this table with the respective step vectors (magnitude and direction).

Yes edge type is not None, then the boundary conditions corresponding to edge type will be used. Yes edge_type = & # 39; bounded & # 39;then the border walks will be zero. Yes edge_type = pacman & # 39;, then the boundary marches will be of magnitude max_edge - min_edge and taken to be in the distant direction of the respective edge.

If anything is not clear, please let me know. Below the code.

import numpy as np
import matplotlib.pyplot as a plt

class IndicialDistributions ():

"" "
The steps are posited in the positive or negative direction according to
to a random number distribution. The methods of this class
return the indices of positive and negative steps given
a type of distribution.

For the moment, only the "normal" distribution type is implemented.
"" "

def __init __ (self, nshape):
"" "
nshape: type 
        "" "
self.nshape = nshape

def get_normal_indices (self, mu, sigma, threshold = None):
"" "
mu: type 
        sigma: type 
        threshold: type  or none
"" "
if the threshold is None:
threshold = mu
random_values ​​= np.random.normal (mu, sigma, size = self.nshape)
pos = (random_values> threshold)
neg = (random_values ​​<threshold)
back pos, neg

def get_binomial_indices (self, p_success, threshold = None):
"" "
p_success: type 
        threshold: type  or none
"" "
raise ValueError ("not yet implemented")

@property
def indicial_function_mapping (auto):
res = {}
res['normal'] = self.get_normal_indices
res['binomial'] = self.get_binomial_indices
return res

def dispatch_indices (self, distribution, ** kwargs):
"" "
distribution: type 
        "" "
available_keys = list (self.indicial_function_mapping.keys ())
if the distribution is not in available_keys:
raise ValueError ("unknown distribution: {}; available distributions: {}". format (distribution, available_keys))
f = self.indicial_function mapping[distribution]
        pos, neg = f (** kwargs)
back pos, neg

BoundaryConditions class ():

"" "
The methods in this class report on the steps taken in the
edges with n dimensions.

From now on, the bounded edges & # 39; and & # 39; pacman & # 39; are implemented.
"" "

def __init __ (auto, steps, moves, max_edge, min_edge):
"" "
steps: type 
        movements: type 
        max_edge: type 
        min_edge: type 
        "" "
self.steps = steps
self.movements = movements
self.max_edge = max_edge
self.min_edge = min_edge

def get_maximum_edge_indices (self):
indices = (self.movements> = self.max_edge)
return hints

def get_minimum_edge_indices (self):
indices = (self.movements <= self.min_edge)
return hints

def apply_maximum_bounded_edge (auto):
indices = self.get_maximum_edge_indices ()
auto.pas[indices] = 0

def apply_minimum_bounded_edge (auto):
indices = self.get_minimum_edge_indices ()
auto.pas[indices] = 0

def apply_pacman_edges (auto):
max_indices = self.get_maximum_edge_indices ()
min_indices = self.get_minimum_edge_indices ()
auto.pas[max_indices] = self.min_edge - self.max_edge
auto.pas[min_indices] = self.max_edge - self.min_edge

def apply_to_dimension (self, list_type):
"" "
edge_type: type 
        "" "
if type_of_board is not None:
If edge_type == & # 39; bounded & # 39 ;:
self.apply_maximum_bounded_edge ()
self.apply_minimum_bounded_edge ()
elif edge_type == & # 39; pacman & # 39 ;:
self.apply_pacman_edges ()
other:
raise ValueError ("unknown edge type: {}; available edge type =" bound "," pacman "or None" .format (edge ​​type))

CartesianRandomWalker class ():

"" "
This class has methods for performing a random walk in n-dimensional space
with the optional use of boundary conditions.
"" "

def __init __ (self, initial_position, nsteps, edge_type = None, max_edges = (), min_edges = ()):
"" "
initial_position: type 
        nsteps: type 
        edge_type: type 
        max_edges: type 
        min_edges: type 
        "" "
self.initial_position = initial_position
self.nsteps = nsteps
self.edge_type = edge_type
self.max_edges = max_edges
self.min_edges = min_edges
self.ndim = len (initial_position)
self.nshape = (self.ndim, nsteps)
self.base = np.zeros (self.nshape) .astype (int)
#self.boundary_crossings = 0
self.current_position = np.array (initial_position)

def __repr __ (auto):
if np.all (self.base == 0):
string = & # 39; Initial position:  t {} & # 39; format (self.initial_position)
other:
string = & # 39; Initial position:  t {}  nNumber of steps:  t {}  nPosition:  t {} & # 39; format (self.initial_position, self.nsteps, self.current_position)
return chain

@property
def (auto) position:
return the tuple (self.current_position)

@property
def (auto) movement:
returns np.cumsum (self.base, axis = 1)

def initialize_steps (self, distribution, delta_steps, ** kwargs):
"" "
distribution: type 
        delta_steps: type 
        "" "
pos, neg = IndicialDistributions (self.nshape) .dispatch_indices (distribution, ** kwargs)
self-basis[pos] = delta_steps[0]
        self-basis[neg] = delta_steps[1]

    def apply_boundary_conditions (self):
if self.edge_type is not None:
for idx in the range (self.ndim):
max_edge, min_edge = self.max_edges[idx], self.min_edges[idx]
                steps = self.base[idx]
                movements = self.movement[idx] + self.initial_position[idx]
                BC = BoundaryConditions (steps, motions, max_edge, min_edge)
BC.apply_to_dimension (self.edge_type)
self-basis[idx, :] = BC.steps

def update_positions (self, distribution, delta_steps = (1, -1), ** kwargs):
"" "
distribution: type 
        delta_steps: type 
        "" "
self.initialize_steps (distribution, delta_steps, ** kwargs)
self.apply_boundary_conditions ()
delta_position = self.movement[:, -1]
        self.current_position + = delta_position

View def (self, ticksize = 7, labelsize = 8, titlesize = 10):
"" "
ticksize: type 
        size of the label: type 
        titleize: type 
        "" "
if self.ndim == 1:
raise ValueError ("not yet implemented")
elif self.ndim == 2:
fig, ax = plt.subplots ()
x_movement = self.movement[0] + self.initial_position[0]
            y_movement = self.movement[1] + self.initial_position[1]
            ax.scatter (* self.initial_position, color = k, label = initial position, marker = x, s = 100)
ax.scatter (* self.current_position, color = "k", label = "Current position", marker = "###", s = 100)
ax.plot (x_movement, y_movement, color = r, alpha = 1/3, label = "Walk Random")
ax.grid (color = "k", linestyle = "," alpha = 0.3)
ax.set_xlabel (x, fontsize = labelize)
ax.set_ylabel (& # 39; Y & # 39 ;, fontsize = labelize)
ax.tick_params (axis = both, labelsize = ticksize)
if self.edge_type is None:
title = r $ {} - $ D Random walk in a Cartesian space & # 39; .format (self.ndim)
elif self.edge_type in (& # 39; bounded & # 39 ;, & quot; pacman & # 39;):
title = & # 39; $ {} - $ D Random walk in Cartesian space  nvia {} Boundary conditions .format (self.ndim, self.edge_type.title ())
ax.set_title (title, fontsize = titlesize)
plt.subplots_adjust (bottom = 0.2)
fig.legend (loc = lower center, mode = expand, fancybox = True, ncol = 3, fontsize = labelize)
plt.show ()
plt.close (fig)
elif self.ndim == 3:
raise ValueError ("not yet implemented")
other:
raise ValueError ("invalid ndim: {}; can only display 1 <= ndim <= 3" .format (self.ndim))

For the moment, only the case in 2 dimensions is visible. I can implement something similar for 1 and 3-dimensional cases, but I am more concerned about the methods than the graph (for the moment). That said, this algorithm can be executed in a 10-dimensional space without the graph. Below is an example of the call.

np.random.seed (327) ## reproduce random results

## initial position
# pos = (50, 50, 50, 50, 50, 50, 50, 50, 50, 50) ## 10-D
pos = (50, 50) ## 2-D
## number of steps to go
nsteps = 100

## random number distribution
## average = 50, spread = 10
## not positive if random number> 50
## not negative if random number <50
## no step if random number = 0
distribution = & # 39; normal & # 39;
kwargs = {mu: # 50, # sigma: 10} # threshold: 50

## conditions to the limits
max_edges = np.array ([60 for element in pos])
min_edges = np.array ([40 for element in pos])
edge_type = None
# edge_type = & # 39; pacman & # 39;
# edge_type = & # 39; bounded & # 39;

RW = CartesianRandomWalker (pos, nsteps, list_type, max_edges, min_edges)
RW.update_positions (distribution, ** kwargs)
print (RW)
RW.view ()

Here is an example of the output of the 2D case:

Starting position: (50, 50)
Number of steps: 100
Current position:   [36 58]

Random walk in 2D

And here is an example of an exit from the 10-D case:

Initial position: (50, 50, 50, 50, 50, 50, 50, 50, 50, 50)
Number of steps: 100
Current position:   [36 58 52 58 38 58 42 78 28 48]

Randomized algorithms – Using exponential penalty functions in constrained nonlinear optimization

Background: penalty functions

Penalty functions to convert a problem of optimization under constraint

begin {equation} begin {split}
text {minimize} quad & f (x) \
text {subject to} quad & g (x) leq 0
end {split} end {equation}

in a problem of optimization without constraint

begin {equation} begin {split}
text {minimize} quad & f (x) + p (g (x), t)
end {split} end {equation}

or $ p: mathbb {R} mapsto mathbb {R} $ is a nondecreasing function and $ t> 0 $ is a "temperature" that increases with the iteration of the optimization procedure, which makes the penalty function more "hard", ie:

$$
lim_ {t to infty} p (g, t) =
begin {cases}
0: & g leq 0 \
infty: & g> 0
end {cases}
$$

The standard choice of a penalty function [1] is something like

$$
p_q (g, t) =
begin {cases}
0: & g leq 0 \
tg ^ 2: & g> 0.
end {cases}
$$

This quadratic sentence has the property that $ p_q (g, t) = 0 $ does not matter when $ g <$ 0. On the other hand, another choice would be a exponential penalty,

$$
p_e (g, t) = exp (tg).
$$

The exponential penalty has good behavior in the limit, but $ p_e (g, t)> 0 $ for $ g <$ 0 and $ t < infty $. This seems bad: consider the case where $ x ^ * $ is a global optimizer of $ f $ but $ g (x ^ *) $ is very close to zero. If there is a solution $ x $ or $ f (x>)> f (x ^ *) $ but $ g (x) ll 0 $, then the exponential penalty solution may be skewed towards $ x $.

Penalty functions with simulated annealing

I use penalty functions with Simulated Annealing (SA) optimization for nonconvex problems and, despite their theoretical drawbacks, I have generally found best results with exponential penalties (solutions with less $ f (x) $).

Simulated annealing works as follows. Let $ E (x, t) = f (x) + p (g (x), t) $:

  1. Start with an initial guess $ x $.
  2. Generate a disturbed value $ x $ by applying a small random change to $ x $
  3. Yes $ E (x); <E (x) $, $ x gets x $
  4. Yes $ E (x); geq E (x) $, $ x gets x $ with decreasing probability $ t (E (x) – E (x)) $
  5. Increase $ t $, go to 2.

Note that SA is usually written with a decreasing temperature, but I modified it to allow the use of the same variable as the temperature of the penalty function.

Is there a reason for exponential penalties to work better in practice? I have seen application documents using the association method with exponential penalties, but I have not found any published justification for using exponential instead of quadratic.

One hypothesis: since the exponential penalty distinguishes possible solutions that almost violate a constraint from those that are far from constraint, it could somehow skew $ x $ to regions where most random disturbances produce a $ x $ this does not violate the constraints, making it "easier" to move between the different attraction basins of local optima. However, I do not know if this is true or how to formalize it.

[1] Luenberger and Ye, "Linear and non-linear programming"