Friday, May 1, 2020

What is economic rent?






Economic rent is a hard term to grasp. According to David Ricardo, rent arises on account of fixed supply of land. But he recognizes other factors which are found in fixed supply in the short term. The additional income earned by these factors in the short-period is similar to rent.

Any payment to a factor of production received in excess of its opportunity cost is economic rent. In our setting, we can safely say that economic rent is used interchangeably with economic profit. Investopedia.com is correct in saying that rent is not the same as normal profit or producer surplus. Normal profit occurs when the economic profit is zero, and profit in excess of the normal profit (i.e., economic profit) is economic rent. Producer surplus is also different from economic profit because it does not take into account of the fixed cost for production (recall that the supply curve is the same as the marginal cost curve, and marginal cost curve does not reflect fixed cost), so economic profit of a producer is less than the producer surplus.

The definition given in the textbook is not wrong, but confusing and unclear. Consumer surplus and producer surplus COULD POTENTIALLY be part of the economic rent. As mentioned above, producer surplus is greater than the economic profit so not all of it is economic rent. Consumer surplus can be switched to economic rent when the producer finds a way to capture it as profit by rent-seeking. When there are no barriers to entry, competition always drives down the economic profit to zero, so economic rent would not exist. Rent-seeking can therefore be considered as ways to mitigate or eliminate competition.

Quasi-rent is a temporary economic rent like returns to a supplier/owner. Quasi-rent differs from pure economic rent in that it is a temporary phenomenon. It can arise from the barriers to entry that potential competitors face in the short run, such as the granting of patents or other legal protections for intellectual property by governments. It can also arise due to entrepreneurial responses to market fluctuation, or due to a lack of real capital to meet near-term increases in demand. In the longer term, however, the opportunity to profit will generate new capital and competition will eliminate the quasi-rent. The joining of opportunism with appropriable quasi-rents (transaction-specific investments) is a leading factor in explaining decisions to vertically integrate. Quasi-rent refers to that additional income which is similar to rent. 


Saturday, July 27, 2019

Difference between standard deviation (SD) and standard error (SE)

Some students confuse the standard deviation and the standard error. Although they have similar names, their meanings are not quite the same. Whether to report the standard deviation or the standard error depends on whether you want to show the dispersion of the SAMPLE or the precision of an ESTIMATE.

The standard deviation measures variability or dispersion of a sample. (Of course, the standard deviation is also used to refer to a parameter of a distribution or a characteristic of a population).

The standard error measures the precision of an estimate.
The standard error of a statistic (usually an estimate of a population parameter) is an estimate of the standard deviation of its sampling distribution.

In particular, the standard error of the mean is the standard deviation divided by the square root of the sample size.

The standard deviation of a sample is a descriptive statistic, whereas the standard error of the mean is descriptive of the random sampling process. The standard deviation of the sample data is a description of the variation in measurements, while the standard error of the mean is a probabilistic statement about how the sample size will provide a better bound on estimates of the population mean, in light of the central limit theorem.

Put simply, the standard error of the sample mean is an estimate of how far the sample mean is likely to be from the population mean, whereas the standard deviation of the sample is the degree to which individuals within the sample differ from the sample mean. If the population standard deviation is finite, the standard error of the mean of the sample will tend to zero with increasing sample size, because the estimate of the population mean will improve, while the standard deviation of the sample will tend to approximate the population standard deviation as the sample size increases.

In practice, if you want to show the precision of an estimate, the 95% confidence interval is often preferred than the standard error (which is, in fact, a sort of 67% confidence interval).











As a side note, estimators have a meaning that is different from estimates.


An estimator is a rule for combining data to produce a numerical value for a population parameter; the form of the rule does not depend on the particular sample obtained.

An estimate is the numerical value taken on by an estimator for a particular sample of data.

An estimator is a random variable, while an estimate is a realization of an estimator based on a sample.


A statistic is a summary of a sample: it is any quantity computed from values in a sample,

We can use the sample mean as the estimator of the population mean. The sample mean (take the sum of all the observation and divide it by the sample size) is an estimator, and the sample mean of a specific sample (which is a statistic of the sample) is an estimate of the population mean.

Friday, October 5, 2018

Python Virtual Environment

A virtual environment is a named, isolated, working copy of Python that that maintains its own files, directories, and paths so that you can work with specific versions of libraries or Python itself without affecting other Python projects. Virtual environments make it easy to cleanly separate different projects and avoid problems with different dependencies and version requirements across components.

Activating a conda environment modifies the PATH and shell variables to point to the specific isolated Python set-up you created.


1. virtualenv

virtualenv is a very popular tool that creates isolated Python environments for Python libraries. It works by installing a bunch of files in a directory (eg: env/), and then modifying the PATH environment variable to prefix it with a custom bin directory (eg: env/bin/). An exact copy of the python or python3 binary is placed in this directory, but Python is programmed to look for libraries relative to its path first, in the environment directory.

Once activated, you can install packages in the virtual environment using pip.


Install virtualenv:
  $ pip install virtualenv

Create a virtual environment:
  $ virtualenv yourenvname

Activate the virtual environment:
  $ source yourenvname/bin/activate
To leave my virtual environment:
  $ deactivate

To install all the required packages specified by the file requirements.txt:
  $pip install -r requirements.txt

If you want to have all the installed packages in a requirements.txt
  $ pip freeze > requirements.txt

To delete a virtual environmrent
 $ rm -r /path/to/yourenvname


2. conda
The conda command is the preferred interface for managing installations and virtual environments with the Anaconda Python distribution.

To see a list of available python versions:
 $ conda search "^python$"

Create a virtual environment with python version x.x for your project
 $ conda create -n yourenvname python=x.x anaconda

To activate or switch into your virtual environment,
 $ source activate yourenvname

To see a list of all your environments,
 $ conda info -e


To install additional packages only to your virtual environment
 $ conda install -n yourenvname [package]

Failure to specify “-n yourenvname” will install the package to the root Python installation.

To end a session in the current environment
 $ source deactivate

Wednesday, January 24, 2018

RMSE vs. R-squared


A well-fitting regression model results in predicted values close to the observed data values. The mean model, which uses the mean for every predicted value, generally would be used if there were no informative predictor variables. The fit of a proposed regression model should therefore be better than the fit of the mean model.
Three statistics are used in Ordinary Least Squares (OLS) regression to evaluate model fit: R-squared, the overall F-test, and the Root Mean Square Error (RMSE). All three are based on two sums of squares: Sum of Squares Total (SST) and Sum of Squares Error (SSE). SST measures how far the data are from the mean and SSE measures how far the data are from the model’s predicted values. Different combinations of these two values provide different information about how the regression model compares to the mean model.
The difference between SST and SSE is the improvement in prediction from the regression model, compared to the mean model. Dividing that difference by SST gives R-squared. It is the proportional improvement in prediction from the regression model, compared to the mean model. It indicates the goodness of fit of the model.
R-squared has the useful property that its scale is intuitive: it ranges from zero to one, with zero indicating that the proposed model does not improve prediction over the mean model and one indicating perfect prediction. Improvement in the regression model results in proportional increases in R-squared.

Friday, October 13, 2017

Power analysis

Power analysis is an important aspect of experimental design. It allows us to determine the sample size required to detect an effect of a given size with a given degree of confidence. Conversely, it allows us to determine the probability of detecting an effect of a given size with a given level of confidence, under sample size constraints. If the probability is unacceptably low, we would be wise to alter or abandon the experiment.

The following four quantities have an intimate relationship:
  1. sample size
  2. effect size
  3. significance level = P(Type I error) = probability of finding an effect that is not there
  4. power = 1 - P(Type II error) = probability of finding an effect that is there
Given any three, we can determine the fourth.


An effect size is a quantitative measure of the strength of a phenomenon. Sample-based effect sizes are distinguished from test statistics used in hypothesis testing, in that they estimate the strength (magnitude) rather than assigning a significance level reflecting whether the magnitude of the relationship observed could be due to chance. The effect size does not directly determine the significance level, or vice versa.

Power analysis can be used to calculate the minimum sample size required so that one can be reasonably likely to detect an effect of a given size. It can also be used to calculate the minimum effect size that is likely to be detected in a study using a given sample size.


An effect size (ES) measures the strength of the result and is solely magnitude based – it does not depend on sample size.  So the effect size is pure – it is what actually was found in the study for the sample studied, regardless of the number of subjects.  But is what was found generalizable to a population? This is where p‐values come into play.  A p‐value gives you the likelihood that what you found is not due to chance.  P‐values very much depend on sample size.

There are three issues with a low power.

1. False negatives
2. Inflated effect size estimates
3. Lower positive predictive value

1. False negatives
The most obvious issue with a low power is the high likelihood of getting false negatives, that is, failing to find an effect that is there. According to the definition, power is 1- P(Type II error). A lower power therefore indicates a high probability of Type II errors (false negatives.)

2. Inflated effect sizes
Cohen's d is often used as a standardized measure of the effect size. It is defined as the difference between two means divided by a standard deviation of the data.

Samples drawn from a population with a given effect size will be distributed around the true effect size. The power of studies does not affect the mean of this distribution, but it affects the shape and areas of significance in the distribution.

The following graph demonstrates the distributions of Cohen's d based on simulation when the true effect size is 0.5 and when the power is 30% and 90% respectively. Note that the distributions are always centered around the true effect size, but the spreads are different -- with a high power, the distribution is more narrowly centered around the true value.  In a sense, with a high power, the effect size you get from the sample is a more accurate estimate of the true effect size.



To understand how power influences the areas of significance, the shaded area in the following graph shows all the effect sizes that corresponds to a statistical test with a p value less than 0.5. Note that with 30% power, it is less likely for a test to be significant, and the values that satisfy the statistical significance are only extreme values. In other words, with a low power size, you only conclude there is a statistically significant effect size when you sample happens to give you an extreme estimate. In this specific case, when you receive an accurate estimate of the true effect size (0.5), it will not pass the significance test.



On the other hand, when the power is 90%, you have a much higher chance to get statistically significant results, and the estimates that pass the test of significance are more likely to be centered around the true effect size.

Suppose we run several studies that investigate a specific effect. When the power is low, then the reported statistically significant results are likely to overestimate the true effect size. If the power is high, then the average estimated effect size from all these studies are much closer to the true effect size.

The following graph reports the average reported Cohen's d as a function of the statistical power based on 10000 runs of simulation with the true Cohen's d as 0.5. Note that when the power is high, the average reported effect size is very close to the true effect size. When the power is low, however, we tend to have an inflated estimate of the effect size.


With a low power, we tend to overestimate the effectiveness of our treatments. It is also difficult to properly power future studies based on past research. 

3. Lower positive predictive value

The positive predictive values are the proportions of positive and negative results in statistics and diagnostic tests that are true positive results. The PPV describes the performance of a diagnostic test or other statistical measure. A high result can be interpreted as indicating the accuracy of such a statistic.

The PPV is defined as

As a function of the significance level (α) and power (1-β), 
Here the odds ratio (OR) represents the odds that the hypothesis is true.

We often do not know the odds of the hypothesis being true when we do a study. But we can look at what the PPV would be for a range of OR and a range of levels of powers. From the following graph we can see that when we have a low power, it is difficult to draw conclusions even from significant studies. This is likely to lead to wasted resources due to following up on false positive studies. 


A study with low statistical power are not likely to detect a true effect. That low power also reduces the likelihood that a statistically significant result reflects a true effect.