Matrix Algebra

A Naive Introduction to Algorithms: Part I



These preliminary algorithmic tours are meant to be a very general introduction to a vast, vast subject. At the same time, I wanted to go far enough and deep enough into the chosen topics so as to provide an overall picture of how at the core, modern statistical algorithms are, in essence, a compendium of very simple, small pieces, which fit together to make a slightly more complex whole. For example, the inverse of the matrix requires one to calculate the determinant (well, usually), which of course requires multiplication, and so on...

For these overviews, I omit detailed proofs, although when warranted, formulas and/or pseudocode is provided. However, most attention is paid to the actual implementation of the algorithm in R. Similarly, math fundamentals/theory are referenced, but not in any great detail. In terms of coverage, I specifically focus on the underpinnings of major statistical algorithms (Linear Algebra/OLS, Optimization/Logistic Regression). Again, it is my hope that this 'tour', so to speak, provides a roadmap for future study, at the very least. After this part is complete, I intend to post more short pieces that consist mainly of algorithms/code with only writing/comments that are truly necessary.

Basic Matrix Operations

As you are probably well aware, the modern computational methods we use today would not be possible without linear algebra. At the most molecular level, most matrix operations are just simple arithmetic that the computer performs over and over again. Again, although nothing is particularly complex, both constructing the iteration and/or recursion scheme and consequent debugging can be tricky given that one missing piece will break everything, or even worse, results will be off in a manner that is difficult to recognize.



Matrix multiplication uses a triple for-loop to sum up products across rows and columns. \[ \textbf{for}\ i=1:m\\\hspace{1cm} \textbf{for}\ j = 1:n\\ \hspace{2cm} \textbf{for}\ k = 1:r\\ \hspace{5cm}M_{i,j} = M_{i,j} + X_{i,k} * Y_{k,j}\\ \hspace{2cm} \textbf{end loop}\ \\ \hspace{1cm} \textbf{end loop}\ \\ \hspace{0cm} \textbf{end loop}\ \]

The subscript i names the row, j names the column, and k handles the dot product. Nevertheless, the ordering of the loops is arbitrary.

    matmult <- function(X,Y) {

     m <- matrix(0,dim(X)[1],dim(Y)[2])
     # Row
     for (i in 1:dim(X)[1]) {
         # Column  
         for (j in 1:dim(Y)[2]) {
             # Column of X or Row of Y
               for (k in 1:dim(Y)[1]) {

                  m[i,j] <- m[i,j] + X[i,k] * Y[k,j]




 x <- matrix(rnorm(16),4,4)
 y <- matrix(rnorm(16),4,4)

##           [,1]       [,2]       [,3]       [,4]
 ## [1,] -2.858235 -2.5435280  1.8303066  1.0208735
 ## [2,]  1.547737 -2.3085681 -0.6122848 -1.3666563
 ## [3,] -3.256425 -0.2429263  0.6441392 -1.0394348
 ## [4,]  1.524830  0.4802570 -0.8023529  0.2774851
x %*% y # base R
##           [,1]       [,2]       [,3]       [,4]
 ## [1,] -2.858235 -2.5435280  1.8303066  1.0208735
 ## [2,]  1.547737 -2.3085681 -0.6122848 -1.3666563
 ## [3,] -3.256425 -0.2429263  0.6441392 -1.0394348
 ## [4,]  1.524830  0.4802570 -0.8023529  0.2774851
# different dims
 z <- matrix(rnorm(8),4,2)

##            [,1]        [,2]
 ## [1,] -0.9093389 -1.03023634
 ## [2,]  1.1929292  0.04823813
 ## [3,] -0.9385204 -2.35411594
 ## [4,]  0.2981690  1.58123597
z %*% x # error in base R
## Error in z %*% x: non-conformable arguments
matmult(z,x) # ditto
## Error in X[i, k]: subscript out of bounds

It is important to note that matrix multiplication is almost always taken care of in a primitive language for speed purposes. R, of course, uses the GNU BLAS/LAPACK/LINPACK etc. libraries which have roughly 50 years of use Also, while the basic matrix multiplication algorithm is \(O(N^3)\), there are other algorithms (e.g., Strassen) that offer some improvement... ...


The determinant is a single value that expresses the overall variability of a matrix. It is used in numerous other matrix algebra applications, including the inverse, which we cover next. \[ det(A) = \sum_{j=1}^n (-1)^{1+j}a_{1j}M_{1j} \]

The basic idea (for a NXN Matrix) is to break everything down into the 2x2 case through minor expansion. The 2x2 case is just the difference of both diagonals multiplied with each other, \(|A| = \begin{bmatrix} a &b\\ c &d\end{bmatrix} = ad - bc \).

D2x2 <- function(X) {

          out <- (X[1,1] * X[2,2] -   X[2,1] * X[1,2])

Then, rows and columns are "blocked" off depending on position The full column and row corresponding to the entry is what is blocked off , and the determinants of the sub-matrices are calculated. These are called "minors" and once calculated, comprise a matrix of minors with each element representing the determinant of the sub-matrix. Reading that last sentence over confuses me and I just wrote it so let's take a look.

matdet <- function(x) {

          D2x2 <- function(X) {

            out <- (X[1,1] * X[2,2] - X[2,1] * X[1,2])

      if (length(x)==1) {


      } else if (any(dim(x)) == 2) {

          dtemp <- D2x2(x)

      } else {

        for (j in 1:dim(x)[2]) {

            multiplier <- ((-1)^(1+j))*(x[1,j]) # alternating sign

          if (j==1) {

              d <- (multiplier*(matdet(x[-1,-j]))) 
    # 'locking' first row by default for simplicity
  } else {       d <- d + (multiplier*(matdet(x[-1,-j])))       }     }   }   return(d) } matdet(x)
## [1] -6.037856
det(x) # base R
## [1] -6.037856

Like most of the algorithms in this intro, there are other/faster See this SO question... ways to compute the determinant, but the simple recursive version is fairly easy to understand. As much as I would like to walk through each operation line by line, space/time considerations don't allow that. Still, if the above is still unclear, I'd recommend following the code either through debugging or taking apart the function in the global environment. Then, look at other algorithms/implementations and compare...

Matrix Inverse

From the determinant, we move to the inverse. There is no such thing as matrix division, but there is multiplication. The inverse can be found by first finding the adjoint matrix, which is just the transpose of the matrix of cofactors This is a common theme, each operation/matrix/whatever has its own name which at first can be overwhelming... , and then dividing (or use inverse of the determinant) each element of the adjoint by the overall determinant. \[ \ Adj(A) = F^T \] \[ \ A^{-1} = \frac{Adj(A)}{Det(A)} \] Take a look below...

inv <- function(x) {

      inversemat <- matrix(0,dim(x)[1],dim(x)[2])

      for (i in 1:dim(x)[1]) {
          for (j in 1:dim(x)[2]) {
              multiplier <- ((-1)^(i+j))
              inversemat[i,j] <- multiplier*(matdet(x[-i,-j]))


      detout <- matdet(x)
      stopifnot(abs(detout) > 1e-04) # stop if determinant is zero

      inversemat <- (1/detout) * t(inversemat) 
  # using base R for transpose here..
  return(inversemat) }

Ordinary Least Squares

So, what was the point of all that? Well, for one, it is now possible to solve the famous OLS equation, \( \hat\beta = (X^TX)^{-1}X^Ty \), from scratch...

X <- as.matrix(mtcars[,5:8],32,3) # just throwing in a few columns
  X <- cbind(1,X) # intercept
  y <- matrix(mtcars[,1]) # matrix so dim() functions in matmult() work correctly

  # steps are arbitrary, just a way to break it down...

  (step1 <- matmult(t(X),X))
##         [,1]      [,2]      [,3]      [,4]    [,5]
  ## [1,]  32.000  115.0900  102.9520   571.160  14.000
  ## [2,] 115.090  422.7907  358.7190  2056.914  54.030
  ## [3,] 102.952  358.7190  360.9011  1828.095  36.558
  ## [4,] 571.160 2056.9140 1828.0946 10293.480 270.670
  ## [5,]  14.000   54.0300   36.5580   270.670  14.000
(step2 <- inv(step1))
##            [,1]        [,2]        [,3]        [,4]        [,5]
  ## [1,] 14.0506300 -1.34446369 -0.37276217 -0.48277552  1.44520301
  ## [2,] -1.3444637  0.23810018  0.07872045  0.01476050 -0.06536778
  ## [3,] -0.3727622  0.07872045  0.08571491 -0.01260577  0.08884587
  ## [4,] -0.4827755  0.01476050 -0.01260577  0.02870522 -0.09624656
  ## [5,]  1.4452030 -0.06536778  0.08884587 -0.09624656  0.50728638
(step3 <- matmult(t(X),y))
##           [,1]
  ## [1,]   642.900
  ## [2,]  2380.277
  ## [3,]  1909.753
  ## [4,] 11614.745
  ## [5,]   343.800
(step4 <- matmult(step2,step3))
##            [,1]
  ## [1,] 10.6166864
  ## [2,]  1.6913239
  ## [3,] -4.4456101
  ## [4,]  0.9980007
  ## [5,] -0.2730062
step1R <- t(X) %*% X
  step2R <- solve(step1)
  step3R <- t(X) %*%  y
  step4R <- step2 %*% step3
##            [,1]
  ## [1,] 10.6166864
  ## [2,]  1.6913239
  ## [3,] -4.4456101
  ## [4,]  0.9980007
  ## [5,] -0.2730062
##                  drat         wt       qsec         vs
  ## 10.6166864  1.6913239 -4.4456101  0.9980007 -0.2730062

QR Decomposition

QR Overview

Although the ability to calculate the inverse of a matrix is all you need for OLS, in actuality, the technique R uses is actually slightly more complicated. Instead, the default least squares fitting method in R is the QR decomposition. There are a number of different types of matrix decompositions A matrix decomposition is where matrix \(A\) can be broken down into multiple other matrices, \(A = CWF\) including LU, Eigen, Cholesky, etc., but QR tends to be the default because it is both a) stable/accurate and b) relatively fast. The R functions qr and qr.coeff See also qr.fitted and qr.resid... allow one to directly utilize this method, but the lm() function runs QR Fortran routines under the hood be default. In fact, this is currently the only method available for lm() or any of its other variants.

Like most statistical algorithms, there are a number of different variants of the QR decomposition and even more sources. For this overview, I closely follow Brown's (2007) Linear Models in Matrix Form. Note that I highly recommend this book as a gentle, but still rigorous text for getting one's feet wet in terms linear algebra/models. Readers in search of a serious, comprehensive guide to matrix algorithms and proofs are directed to Golub and Van Loan (2013) or one of the SIAM publications.. discussion of the QR decomposition/algorithm. Although this is one of the rare books that actually presents R code (as opposed to MATLAB) at the end of each section, but for learning/challenge/fun(??) reasons, I code them directly from formulas so there will undoubtedly be differences in the approach.

Like other decompositions (see sidenote #5), the QR factors a matrix into Q, an orthonormal matrix "Orthonormal (orthogonal) matrices are matrices in which the columns vectors form an orthonormal set (each column vector has length one and is orthogonal to all the other colum vectors" , and R, an upper-triangular matrix. Moreover,

An orthonormal matrix returns an identity matrix when it is premultiplied by its transpose, and an upper triangular matrix has 0's below the main diagonal. These features make the QR decomposition a highly efficient and accurate method for solving a variety of statistical problems.

Brown, 2014

Householder Transform

The QR decomposition can be performed with a number of different orthogonalization techniques. For dense matrices Givens rotations require fewer operations for sparse matrices... , Householder Dr. Alston Scott Householder, pretty smart dude.. transformations, tend to produce highly reliable results,

Householder transformations are used to selectively zero out blocks of entries in vectors or columns of matrices in a manner that is extremely stable with respect to round-off error.

Burden & Faire, 2011

The basic formula is, \( H = I − 2uu^t\).

In terms of the process, following Brown (2014), Burden and Faire (2011) Basic numerical computing textbook..., Golub and Van Loan (2013), etc., one first creates an identity matrix with k+1 (includes intercept) and then calculate \(u\) via the following ratio, \[ u=\frac{v\pm\|v\|p_{i}}{\|(v\pm\|v\|p_{i})\|} \] where the plus-minus in the numerator depends on whether the first non-zero element is greater than (plus), or less than (minus), zero. The ratio itself consists of the column-vector of the original matrix plus(minus) the Euclidean norm of the same vector times the corresponding column of the identity matrix over the Euclidean norm of the numerator. If this leads to a 'wtf!!??' moment, implementation in R code may add clarification.

# Euclidean norm

     eucnorm <- function(x) (x <- sqrt(sum(x^2)))

       calculate_u <- function(A,p,i) {

         # don't forget to add the drop=FALSE (like I did at first!!)  
         v <- A[,i,drop=FALSE]

         if (i>1) v[1:i-1] <- 0 # adding zeros

         # finds the first non-zero element
         x <- Find(function(x) x != 0, x = v)

         if (x > 0) {  # if positive, then addition

                 num <- v + (eucnorm(v) * p[,i,drop=FALSE])

                 denom <- eucnorm(v + eucnorm(v) * p[,i,drop=FALSE])

                 u <- num / denom


         } else {

           # if negative, then subtraction

                 num <- v - (eucnorm(v) * p[,i,drop=FALSE])

                 denom <- eucnorm(v - eucnorm(v) * p[,i,drop=FALSE])

                 u <- num / denom




Each resulting \(H_{p}\) is in turn used to pre-multiply the original matrix where already transformed columns are set to zero to get \(R\). \(Q\) is found by the multiplicative product of \(\prod P_{i}\). Again, code makes this considerably more clear...

M <- matrix(rnorm(16),4,4)

  QR <- function(A) {

      # latter condition below isn't necessary, but makes things easier for common case..

      stopifnot(is.matrix(A) | (dim(A)[2] >= dim(A)[1]))

      I <- diag(dim(A)[1])

      Hlist <- list()
      Rlist <- list()
      QRlist <- list()

      for (i in 1:dim(A)[2]) {

          if (i==1) {

              u <- calculate_u(A,I,i)

              Hlist[[i]] <- I - 2 * matmult(u,t(u))

              Rlist[[i]] <- matmult(Hlist[[i]],A)

          } else {

          u <- calculate_u(Rlist[[(i-1)]],I,i)

          Hlist[[i]] <- (I - 2 * matmult(u,t(u)))
          Rlist[[i]] <- matmult(Hlist[[i]],Rlist[[(i-1)]])



      Q <- Reduce(matmult,Hlist)  ## yes, cheating with Reduce() here...
      R <- Rlist[[dim(A)[2]]]

      QRlist <- list(Q=Q,R=R)


  (out <- QR(M))
## $Q
  ##            [,1]       [,2]       [,3]       [,4]
  ## [1,] -0.3382630  0.2907179  0.5490472 -0.7068298
  ## [2,] -0.0948519 -0.3569862 -0.6820139 -0.6312060
  ## [3,]  0.7592288  0.5737387 -0.1675652 -0.2575218
  ## [4,]  0.5478621 -0.6773979  0.4531293 -0.1888204
  ## $R
  ##               [,1]          [,2]          [,3]       [,4]
  ## [1,] -1.362792e+00  1.230086e+00  1.155561e+00 -0.8892821
  ## [2,]  1.221896e-16 -1.287082e+00 -1.064953e+00 -0.4146394
  ## [3,] -6.577927e-17  1.750513e-16 -2.292122e+00  1.1328554
  ## [4,] -3.760067e-18  2.423225e-16  4.440892e-16 -1.5915067

So now what? Well, for one, you can reconstruct the original matrix nearly perfectly...

##            [,1]       [,2]       [,3]       [,4]
  ## [1,]  0.4609820 -0.7902707 -1.9589672  1.9271836
  ## [2,]  0.1292634  0.3427946  1.8338251  0.4643161
  ## [3,] -1.0346706  0.1954678  0.6504099 -0.6930427
  ## [4,] -0.7466218  1.5457845  0.3158573  0.6075107
##            [,1]       [,2]       [,3]       [,4]
  ## [1,]  0.4609820 -0.7902707 -1.9589672  1.9271836
  ## [2,]  0.1292634  0.3427946  1.8338251  0.4643161
  ## [3,] -1.0346706  0.1954678  0.6504099 -0.6930427
  ## [4,] -0.7466218  1.5457845  0.3158573  0.6075107
## [1] TRUE

The QR algorithm can also easily be extended to obtain the eigenvalues of a matrix, which is, in fact, in many cases its main purpose. What is an eigenvalue you ask?

To explain eigenvalues, we first explain eigenvectors. Almost all vectors change di-rection, when they are multiplied by A. Certain exceptional vectors x are in the same direction as Ax. Those are the "eigenvectors". Multiply an eigenvector by A, and the vector Ax is a number lamda times the original x.

The basic equation is \( Ax = \lambda x \). The number \(\lambda\) is an eigenvalue of A.

The eigenvalue lambda tells whether the special vector x is stretched or shrunk or reversed or left unchanged-when it is multiplied by A.

In case you couldn't tell, defining eigenvalues(vectors) in an inuitive sense without relying on the geometric interpretation or mathmatical operations is rather difficult, if not impossible. For the completely uninitiated, I'd recommend watching several YouTube videos If the author's, uh, enthusiasm isn't too much for you try this channel... and then checking out any solid linear algebra textbook See Banerjee and Roy (2014), Gruber (2013), or even the cheekily titled No Bullshit Guide to Linear Algebra by Ivan Savov (2016)... . However, once you venture into a specific domain, they can take on a more concrete meaning. For example, in my field, quantitative psychology, eigenvalues play a large role in dimensional reduction techniques like factor analysis and PCA. In this case, when performing a factor analysis on a number of psychometric items, one would look for the largest eigenvalues as they hold more information than those that are smaller (this, alas, is a massive subject in its own right). Before I demonstrate code, it's also important to note that eigenvalues are usually, but not always, computed for symmetric matrices. However, in the latter case, things get dicey quick and software like R resorts to the use of complex numbers for approximations..

Here, in the following code, we get small taste of iteration until a certain tolerance level is reached, something that is a fundamental aspect for optimization algorithms.

QR_alg <- function(A, iter) {

      QR_i <- QR(A)
      RQ <- QR_i[[2]] %*% QR_i[[1]]
      e <- rep(1, dim(RQ)[2])

      while (i < iter) {

          i <- i + 1

          QR_i <- QR(RQ)
          RQ <- QR_i[[2]] %*% QR_i[[1]]

          x <- isTRUE(all.equal(diag(RQ),e))

          if (x) {

          e <- diag(RQ)


      out <- list(RQ=RQ,eigenvals=e,iter=i)


  head(X) # recall our subset from mtcars from earlier
##                     drat    wt  qsec vs
  ## Mazda RX4         1 3.90 2.620 16.46  0
  ## Mazda RX4 Wag     1 3.90 2.875 17.02  0
  ## Datsun 710        1 3.85 2.320 18.61  1
  ## Hornet 4 Drive    1 3.08 3.215 19.44  1
  ## Hornet Sportabout 1 3.15 3.440 17.02  0
  ## Valiant           1 2.76 3.460 20.22  1
X_SS <-  matmult(t(X),X) # sum of squares

## [1] 1.106900e+04 4.103495e+01 1.077402e+01 2.294991e+00 6.965035e-02
# as far as I know, no eigenvalues via QR in R..

  eigen(X_SS,only.values = TRUE)
## $values
  ## [1] 1.106900e+04 4.103495e+01 1.077402e+01 2.294991e+00 6.965035e-02
  ## $vectors
  ## NULL

Note that this code could Read should... be extended to include shifts which alter the diagonal of A \(RQ + sI\) by subtracting values that appear after i iterations from the values of iteration \(i+1\).

Finally, we can get our least squares estimates via, \[ b = R^{-1}Q^{T}y \] It is important to mention that you will need to 'cull' row/columns based on the QR() code provided above in order to make R symmetric, or, in other words, keep only as many rows as the non-zero upper triangular part. Similarly, for Q, take just the number columns (plus vector of one's) as there were variables in the original matrix. This is the 'skinny QR'. See here...

X <- as.matrix(mtcars[,5:8],32,3)
  X <- cbind(1,X)
  y <- matrix(mtcars[,1])

  Q <- QR(X)[[1]]
  R <- QR(X)[[2]]

  step1QR <- inv(R[1:5,1:5]) # making R symmetric as it 'should be'
  step2QR <- t(Q[,1:5]) # other columns are rubbish
  step3QR <- matmult(step1QR,step2QR)

##            [,1]
  ## [1,] 10.6166864
  ## [2,]  1.6913239
  ## [3,] -4.4456101
  ## [4,]  0.9980007
  ## [5,] -0.2730062,y)$coefficients
##                  drat         wt       qsec         vs
  ## 10.6166864  1.6913239 -4.4456101  0.9980007 -0.2730062

Well, that's more than enough for now. This took quite a while to do, despite its incomplete nature. Moreover, numerous little bugs popped up for the more complicated algorithms when creating examples (and there are probably more I haven't found!), as, afterall, the 'devil is in the details'. One of my favorite things with the heavy duty linear algebra and optimization textbooks are the warnings at the front that code should not be "used as-is" in production applications and most certainly not where lives are at stake! Makes me feel special reading big boy(girl) books...




2019-01-21 00:36:00
Buy Ocuvir Cream [url=]cialis online[/url] Finasteride Farmaco Propecia Viagra Pour Elle Uso Viagra Para Mujeres Zitromax On Line [url=]cialis 5mg best price[/url] Kamagra Superactive Amoxicillin And Yeast Infection Order Diflucan Online No Prescription [url=]cialis overnight shipping from usa[/url] Lowest Price Flagyl Low Price My Pharmacy 365 Order Now Doryx In Internet Buy Fincar On Line Comprare Levitra In Italia [url=]antabuse[/url] Buy Valtrex 1000 Mg Discount Isotretinoin Buy [url=]levitra and cialis online[/url] 100mg Viagra For Sale Order Metronidazole 200mg Online Canada Cheap Cialis And Viagra
Cialis Coupon
2019-04-20 19:55:00
Side effects of drinking alcohol and taking Coupon for Cialis? <a href=" ">Cialis Generic</a> Cheap Cialis no script. Best online site for Cialis Coupon reviews. <a href=" ">Coupon for Cialis</a> Can you drink alcohol and take Cialis Coupon?
2019-04-26 15:03:00
Side Effects Keflex [url=]cialis cheapest online prices[/url] Viagra Opinione
buy viagra online
2019-05-29 17:16:00
Thanks to GM I had sex for the first time in two months. <a href="">generic viagra</a> We want you to be fully satisfied with every item that you purchase from www.
order viagra online
2019-06-05 11:45:00
Abuse is Bad Bad chronic will help you value of cookie will right now here in. <a href="">where to buy viagra</a> Most designer clothing labels are actually produced in India and China as well, so this is not an issue.
2019-09-10 03:58:00
Viagara For Men Levitra 10 Maison De La Gendarmerie [url=]viagra[/url] Chemical Properties Of Amoxicillin
2019-10-19 05:29:00
2020-03-10 20:51:00
buy viagra generic viagra <a href="">viagra 100mg </a> buy viagra buy viagra bh88апрпhfa

viagra online generic viagra [url=]viagra online [/url] viagra online viagra 100mg
2020-05-24 07:29:00
canada pharmaceuticals online generic [url=]online pharmacy [/url]
pharmacy in canada <a href="">canada discount drug </a>
aarp recommended canadian pharmacies
2020-06-02 06:31:00
custom my essay [url=]custom essay writing toronto [/url]
best custom essay website <a href="">write essays for me </a>
someone write my essay for me
2020-06-02 07:52:00
kamagra silagra generic cialis [url=]generic cialis tadalafil [/url]
rugs that dont mix cialis <a href="">cialis pill </a>
raynauds cialis
2020-06-02 14:03:00
essay writer program [url=]online essay writers wanted [/url]
best custom essay writing <a href="">essay typer </a>
cheap essay writers
2020-06-02 15:59:00
qual o mais eficaz viagra ou cialis [url=]pharmacy cialis generic online [/url]
cialis kamagra kaufen <a href="">cialis without a doctor prescription </a>
cialis ohne rezept erfahrungen
2020-06-02 21:23:00
writing custom essays [url=]essay for me [/url]
write my essays <a href="">academic essay writer </a>
someone write my essay
2020-06-03 04:41:00
best custom essay writers [url=]custom essays toronto [/url]
write my essay for me cheap <a href="">persuasive essay </a>
custom essays cheap
2020-06-03 12:09:00
custom essay paper [url=]write my essay [/url]
professional essay writers <a href="">write my essay for money </a>
top 10 essay writers
2020-06-03 15:38:00
cialis ja viagra [url=]cialis generico [/url]
viagra or cialis which is stronger <a href="">cialis tadalafil </a>
cialis commentscgi generic mt tadalafil
2020-06-03 19:24:00
essay writers canada [url=]write my essays [/url]
custom application essay <a href="">essay rewriter </a>
best custom essay
2020-06-03 23:17:00
generic levitra cialis viagra [url=]tadalafil 5mg [/url]
cialis cheap canada blue mountain <a href="">buy cialis </a>
buy cialis soft online get prescription
2020-06-04 02:32:00
custom law essay [url=]write essays for me [/url]
custom essay paper <a href="">professional essay writers review </a>
custom essays no plagiarism
2020-06-04 07:24:00
navarro pharmacy miami [url=]canadian pharcharmy [/url]
walmart pharmacy online <a href="">pharmacy online </a>
pharmacy online no prescription
2020-06-04 09:53:00
essay writer online [url=]who can write my essay [/url]
write my essay cheap <a href="">magic essay writer </a>
professional essay writers
2020-06-04 15:06:00
cialis 20 mg filmtabletten bestellen [url=]buy cialis [/url]
subaction showcomments cialis archive online <a href="">buy cialis generic </a>
only here cialis pills
2020-06-04 16:04:00
walmart pharmacy viagra [url=]best canadian online pharmacy [/url]
canada drugs pharmacy <a href="">apollo pharmacy online </a>
canadian pharmacy online
2020-06-04 17:11:00
custom essay writing cheap [url=]custom essays online [/url]
best custom essay writing <a href="">cheap essay writer </a>
quality custom essays
2020-06-04 22:44:00
cellules sp cialis es [url=]cialis 5 mg [/url]
interaction between viagra and cialis <a href="">tadalafil generic </a>
cialis quick tab
2020-06-05 00:16:00
custom essay writer [url=]what should i write my essay on [/url]
custom writing essays <a href="">customized essays </a>
the best essay writers
2020-06-05 00:17:00
best canadian online pharmacies [url=]online medicine to buy [/url]
online pharmacies uk <a href="">canadian drugs online pharmacy </a>
pharmacy online shopping
2020-06-05 07:35:00
best custom essay site [url=]custom essays cheap [/url]
essay writer <a href="">essay for me </a>
persuasive essay writer
2020-06-05 09:01:00
indian pharmacy [url=]pharmacy uk [/url]
pharmacy in canada <a href="">pharmacy uk </a>
24 hour pharmacy
2020-06-05 15:08:00
custom essay meister [url=]best essay writer [/url]
instant essay writer <a href="">custom essays </a>
custom law essays
2020-06-05 18:03:00
online pharmacy busted [url=]pharmacy online cheap [/url]
canada online pharmacies <a href="">canadian prescriptions online </a>
canadian pharmacy online viagra
2020-06-05 22:30:00
write my essay reviews [url=]essay rewriter [/url]
english essay writers <a href="">write a paper for me </a>
custom essay paper
2020-06-05 23:17:00
buy cialis in south africa online [url=]cialis [/url]
qual o melhor viagra cialis levitra e vivanza <a href="">cheap cialis </a>
cialis canadian epharmacy
2020-06-06 02:48:00
canadian drugs pharmacy [url=]pharmacies in canada [/url]
drugstore online shopping <a href="">canadian drugs </a>
prescription drugs from canada
2020-06-06 05:53:00
custom essays writing [url=]best custom essay [/url]
custom essays online <a href="">essay writer </a>
custom essay uk
2020-06-06 07:21:00
cialis interactions [url=]tadalafil 20mg [/url]
cilas <a href="">cialis tadalafil </a>
cialis en viagra combineren
2020-06-06 11:33:00
online pharmacies [url=]order medicine online [/url]
online pharmacies uk <a href="">discount pharmacies </a>
canadian pharmacies-24h
2020-06-06 13:18:00
online custom essays [url=]cheap custom essays online [/url]
personal essay writers <a href="">do my essay for me </a>
essay writer reviews
2020-06-06 15:22:00
can you cut cialis 20 mg in half [url=]cialis 5 [/url]
what is the cost difference between viagra and cialis <a href="">cialis 20mg </a>
canadian cialis
2020-06-06 20:08:00
navarro pharmacy miami [url=]canadian pharmaceuticals online [/url]
pharmacy drugstore online <a href="">online canadian pharcharmy </a>
online pharmacies canada
2020-06-06 20:38:00
order custom essays [url=]write an essay [/url]
custom english essays <a href="">top 10 essay writers </a>
essay writer funny
2020-06-07 04:10:00
professional essay writers [url=]custom essay company [/url]
top 10 essay writers <a href="">essay on </a>
custom essays toronto
2020-06-07 04:51:00
24 hour pharmacy [url=]online medicine tablets shopping [/url]
pharmacy on line <a href="">discount pharmacy </a>
walmart pharmacy online
2020-06-07 07:26:00
cialis copy [url=]cialis buy generic [/url]
cialis discount coupon <a href="">cialis </a>
cialis for sale
2020-06-07 11:35:00
online essay writer [url=]do an essay for me [/url]
best online essay writers <a href="">hire essay writer </a>
professional grad school essay writers
2020-06-07 18:49:00
magic essay writer [url=]essay [/url]
custome essay <a href="">custom essay meister </a>
someone to write my essay
2020-06-08 02:08:00
online essay writers wanted [url=]custom essay writing [/url]
my custom essay <a href="">personal essay </a>
academic custom essays
2020-06-08 06:36:00
pharmacy online cheap [url=]canadian government approved pharmacies [/url]
canadian viagra generic pharmacy <a href="">pharmacy online cheap </a>
viagra pharmacy 100mg
2020-06-08 07:12:00
australia in sale cialis soft [url=]buy generic cialis [/url]
cialis viagra sale <a href="">tadalafil 20 mg </a>
viagra vs cialis effectiveness
2020-06-08 09:33:00
customized essays [url=]professional essay writer [/url]
custom writing essays <a href="">us essay writers </a>
college essay writers
2020-06-08 15:22:00
shoppers drug mart pharmacy [url=]online pharmacy canada [/url]
london drugs canada <a href="">pharmacies </a>
pharmacies in canada
2020-06-08 16:55:00
custom essay station [url=]college essay [/url]
online essay writers <a href="">write my essay </a>
cheap custom essay papers
2020-06-08 23:08:00
qual melhor viagra levitra cialis [url=]buy cialis online [/url]
buy cialis soft pay with paypal <a href="">cialis buy generic </a>
cialis high blood pressure
2020-06-09 00:15:00
custom order essays [url=]essay on [/url]
cheap custom essay <a href="">essay for me </a>
custom essays review
2020-06-09 07:10:00
essays custom [url=]custom essay writing cheap [/url]
custom essay cheap <a href="">college essay </a>
the best essay writers
2020-06-09 14:58:00
write my essay for money [url=]write my essay for me [/url]
custom essay cheap <a href="">essay writer </a>
custom essays usa
2020-06-09 22:28:00
custom admission essay [url=]application essay [/url]
best essay writer <a href="">write a essay for me </a>
custom essays for sale
2020-06-10 06:03:00
great essay writers [url=]pay to write my essay [/url]
essay writer generator <a href="">college application essay writers </a>
write my essay cheap
2020-06-10 07:51:00
viagra cialis pret [url=]generic cialis online [/url]
viagra and cialis samples <a href="">cialis 20 </a>
cialis combineren met viagra
2020-06-10 13:34:00
essay writer online [url=]professional college application essay writers [/url]
pay someone to write my essay <a href="">write my college essay for me </a>
who can write my essay
2020-06-10 15:41:00
cialis spendere poco [url=]cialis without a doctor prescription [/url]
cialis generika eu apotheke <a href="">cialis online </a>
how often cialis soft tabs
2020-06-10 20:55:00
cheap custom essay papers [url=]essay writer [/url]
online custom essays <a href="">custom essays </a>
essay writer generator
2020-06-10 23:32:00
free sample generic cialis pills [url=]price cialis [/url]
cheap cialis pillstore <a href="">cialis online </a>
cialis generika in niederlande
2020-06-11 04:36:00
famous essay writers [url=]online essay writers wanted [/url]
writing custom essays <a href="">write an essay </a>
academic essay writer
2020-06-11 12:24:00
custom written essays [url=]online essay writer [/url]
write my essay for me cheap <a href="">custom essay uk </a>
custom admission essay
2020-06-11 19:48:00
quality custom essays [url=]essay on [/url]
custom essay writing reviews <a href="">order custom essay </a>
paid essay writers
2020-06-12 03:20:00
cheapest custom essays [url=]custom essay [/url]
academic custom essays <a href="">best custom essay </a>
custom essay writing cheap
2020-06-12 08:11:00
cialis pills without a prescription [url=]generic cialis tadalafil [/url]
cialis bph clinical trial <a href="">tadalafil cialis </a>
cialis 10 mg online bestellen
2020-06-12 10:50:00
custom essay writing [url=]pay for essay [/url]
custom essay company <a href="">college essay writer </a>
best essay writer
2020-06-12 18:13:00
essay writers needed [url=]essay typer [/url]
write my essay cheap <a href="">professional essay writer </a>
essay writer reviews
2020-06-13 01:24:00
custom essay order [url=]personal essay [/url]
quality custom essays <a href="">write my essay for me </a>
write my essay org
2020-06-13 08:42:00
online essay writers [url=]write essays for me [/url]
essay writer <a href="">best essay </a>
top custom essays
2020-06-13 15:30:00
cost of cialis vs viagra vs levitra [url=]cialis [/url]
buy brand name cialis <a href="">cialis generic online </a>
cialis outdated
2020-06-13 16:07:00
order custom essay online [url=]write my college essay for me [/url]
custom admission essay <a href="">college essays </a>
pay someone to write my essay
2020-06-13 23:25:00
order custom essays [url=]write essays for me [/url]
write custom essays <a href="">essay rewriter </a>
professional college essay writers
2020-06-14 06:47:00
custom essay meister [url=]write my essay custom writing [/url]
custom essays for cheap <a href="">pay for essay </a>
write custom essays
2020-06-14 14:21:00
essay writer program [url=]college essay [/url]
custom essays toronto <a href="">custom essay writing </a>
custom order essays
2020-06-14 21:48:00
how to be a good essay writer [url=]custom essay paper writing [/url]
custom essay writer <a href="">college essay </a>
essay writer online
2020-06-15 04:56:00
write my essay reviews [url=]custom written essay [/url]
good essay writers <a href="">college essays </a>
fast custom essay
2020-06-15 08:58:00
write my essay custom writing [url=]an essay [/url]
custom application essay <a href="">academic essay writers </a>
urgent custom essays
2020-06-15 12:56:00
essay writers review [url=]write a essay for me [/url]
essay writers wanted <a href="">custom essay writing </a>
custom essays usa
2020-06-15 16:42:00
best custom essay [url=]what should i write my essay on [/url]
professional essay writer <a href="">college essay writers </a>
essay writer reddit
2020-06-15 20:28:00
essay writers wanted [url=]an essay [/url]
custom essay order <a href="">pay for essay </a>
custom law essay
2020-06-16 00:16:00
customer essay [url=]how to be a better essay writer [/url]
automatic essay writer <a href="">cheap essay </a>
custom essays toronto
2020-06-16 00:18:00
cialis comments cgi generic mt tadalafil [url=]cialis online [/url]
cialis tubs <a href="">generic cialis buy </a>
cialis v viagra uk
2020-06-16 04:02:00
customized essay writing [url=]professional custom essays [/url]
professional custom essays <a href="">custom essay </a>
college essay writer
2020-06-16 07:50:00
cheap custom essays [url=]professional essay writers [/url]
custom english essays <a href="">do my essay </a>
custom made essays
2020-06-16 08:22:00
cialis shop ervaringen [url=]cialis online [/url]
natural substitute cialis soft <a href="">buy cialis pills </a>
indian generic cialis soft
2020-06-16 11:38:00
essay writers wanted [url=]custom law essays [/url]
cheap essay writer <a href="">essay rewriter </a>
essay writers online cheap
2020-06-16 15:25:00
essay writers for pay [url=]order custom essays online [/url]
custom essay writing <a href="">write my essay reviews </a>
write my essay org
2020-06-16 19:16:00
essay writers net [url=]best custom essay website [/url]
essay writers needed <a href="">application essay </a>
custom essay station
2020-06-16 23:01:00
someone write my essay for me [url=]college application essay [/url]
custom essay toronto <a href="">write my college essay for me </a>
what is the best custom essay site
2020-06-17 02:43:00
cheapest essay writers [url=]custome essay [/url]
essay on old custom <a href="">essay writers toronto </a>
custom essay paper writing
2020-06-17 08:24:00
mieux cialis ou viagra [url=]cialis tablets [/url]
works best viagra cialis levitra <a href="">cialis online </a>
cialis promotion
2020-06-17 10:51:00
Precios Propecia [url=]Cialis[/url] Suprax <a href=>Generic Cialis</a> Nsu Treatment Antibiotic Azithromycin
2020-06-18 08:35:00
does medicare pay for viagra and cialis [url=]generic cialis 20 mg [/url]
cialis once a day kopen <a href="">cialis price </a>
cialis ingrediants
2020-06-19 03:52:00
ed meds online canada <a href=" ">is it illegal to buy prescription drugs online</a> pet meds without vet prescription
2020-06-19 05:00:00
buy prescription drugs without doctor <a href=" ">how to get prescription drugs without doctor</a> cheap pet meds without vet prescription
2020-06-19 07:32:00
cialis induced tinnitus [url=]buy generic cialis [/url]
viagra e cialis effetti collaterali <a href="">buy online cialis </a>
qual melhor viagra cialis
2020-06-19 15:24:00
cialis 5mg cheap [url=]viagra vs cialis [/url]
cialis trial pack $38 <a href="">cialis generic </a>
imprimeurs sp cialis s
2020-06-21 06:00:00
female use of cialis [url=]cialis [/url]
cialis generico super active 20mg <a href="">cialis tadalafil </a>
cialis two bathtubs fear
2020-06-21 13:39:00
cialis c5 review [url=]cialis online pharmacy [/url]
cialis for daily use review <a href="">cialis generic buy </a>
subaction showcomments cialis sale posted
2020-06-21 21:05:00
cialis 5 mg precio farmacia [url=]generic cialis online [/url]
1cialis levitra viagra vs vs <a href="">cialis generic </a>
uk cialis soft prices
2020-06-22 00:33:00
comfortis without vet prescription [url=]prescription drugs online[/url] cheapest real cialis
buy cialis 5 mg canada
2020-06-22 04:41:00
cialis and bent penis [url=]tadalafil generic [/url]
can you buy cialis soft in brazil <a href="">generic cialis tadalafil </a>
free sample offer for cialis
2020-06-22 20:21:00
buy anti biotics without prescription <a href=" ">buy prescription drugs from canada</a> pain medications without a prescription
2020-06-23 03:50:00
2cialis levitra viagra vs vs [url=]online buy cialis [/url]
cialis for daily use reviews <a href="">online cialis generic </a>
viagra dosage vs cialis dosage
2020-06-23 19:43:00
cialis generic availability <a href=" ">buy cialis</a> cialis generic
2020-06-24 10:06:00
how does cialis work <a href=" ">buy generic cialis</a> cialis
generic cialis bitcoin [url=]generic cialis[/url] canada viagra
2020-06-24 12:07:00 viagra without doctor prescription
2020-06-24 21:53:00
cialis 20 mg best price <a href=" ">cialis</a> generic viagra
2020-06-25 01:49:00
we recommend cheapest cialis [url=]cialis tadalafil [/url]
prescription cialis on line <a href="">tadalafil 20mg </a>
viagra cialis levitra online australia
2020-06-25 06:25:00
buy prescription drugs <a href=" ">pain meds online without doctor prescription</a> legal to buy prescription drugs without prescription
2020-06-25 07:51:00
good essay writers [url=]persuasive essay [/url]
cheap essay writers <a href="">the best essay writers </a>
professional essay writers for hire
2020-06-25 11:32:00
my custom essay [url=]essay for me [/url]
how to be a good essay writer <a href="">best custom essay </a>
academic essay writer
2020-06-25 15:06:00
custom essay cheap [url=]custom essay order [/url]
write my essay for money <a href="">writing custom essays </a>
best custom essay sites
2020-06-25 17:25:00
i cialis better than viagra [url=]generic cialis 20 mg [/url]
cialis commentscgi mt tadalafil <a href="">cialis generique </a>
dose low viagra cialis drug
2020-06-25 18:39:00
the best essay writer [url=]best custom essay [/url]
how to be a better essay writer <a href="">best essay writers </a>
essay writer
2020-06-25 22:09:00
cheapest essay writers [url=]top custom essays [/url]
customized essays <a href="">custom application essay </a>
automatic essay writer
2020-06-25 22:43:00
prescription meds without the prescriptions <a href=" ">pain meds without written prescription</a> buy prescription drugs from canada cheap
2020-06-26 01:40:00
how to be a better essay writer [url=]college essays [/url]
cheap essay writers <a href="">custom college essays </a>
custom essay cheap
2020-06-26 05:22:00
someone to write my essay [url=]essay [/url]
essay writer funny <a href="">essay writers </a>
cheapest essay writers
2020-06-26 17:53:00
cialis at a discount price <a href=" ">generic viagra</a> and <a href=" ">buy cialis</a> and <a href=" ">cheap cialis</a> and <a href=" ">viagra without doctor prescription</a> and <a href=" ">cialis generic</a> and <a href=" ">buy cialis</a> viagra without doctor prescription
2020-06-26 20:07:00
prescription drugs online without doctor <a href=" ">prescription drugs online</a> pet meds without vet prescription
2020-06-27 04:49:00
does cialis lower your blood pressure <a href=" ">generic cialis</a> and <a href=" ">buy cialis</a> and <a href=" ">buy cialis</a> and <a href=" ">generic cialis</a> and <a href=" ">buy cialis</a> and <a href=" ">cialis</a> cialis without doctor prescription
2020-06-28 12:42:00
online ed meds <a href=" ">cialis coupon</a> generic cialis
2020-06-29 06:14:00
ed prescription drugs <a href=" ">cialis</a> cialis price
2020-06-29 21:01:00
when is the best time to take cialis <a href=" ">cialis coupon</a> and <a href=" ">cialis 20mg</a> and <a href=" ">cialis</a> and <a href=" ">buy generic cialis</a> and <a href=" ">buy generic cialis</a> and <a href=" ">cialis</a> buy cialis online canadian <a href=" ">buy cialis online</a> and <a href=" ">cialis coupon</a> and <a href=" ">cialis online</a> and <a href=" ">cialis 20mg</a> and <a href=" ">cialis</a> and <a href=";u=443895 ">buy cialis</a> generic cialis
2020-06-29 23:20:00
fastest delivery of cialis buying online <a href=" ">cialis 20mg</a> and <a href=" ">cialis 20mg</a> and <a href=" ">cialis coupon</a> and <a href=";u=33170 ">cialis online</a> and <a href=" ">cialis online</a> and <a href=" ">cialis coupon</a>;u=87205 buy cialis
2020-06-30 06:32:00
generic cialis without prescription <a href=" ">cialis</a> and <a href=" ">buy cialis</a> and <a href=" ">cialis 20mg</a> and <a href=" ">cialis coupon</a> and <a href=" ">cialis 20mg</a> and <a href=" ">cialis</a> cialis price
2020-07-01 21:42:00
show cialis working <a href=" ">buy cialis</a>
2020-07-02 19:26:00
fastest delivery of cialis buying online <a href=" ">samples of cialis</a>
2020-07-03 00:29:00
does cialis lower blood pressure <a href=" ">cialis tadalafil</a>
cialis 20mg:
2020-07-03 03:17:00 - price of cialis
[url=]cialis vs viagra[/url]
2020-07-03 05:13:00
cialis professional <a href=" ">buy cialis</a> - cialis samples request
[url=]cialis vs viagra[/url]
2020-07-03 17:02:00
<a href=" ">5mg cialis</a> - how does cialis work
[url=]cialis tadalafil[/url]