Added By: Feedage Forager | |
Language: English | |
Tags: | |
economic expectations good ignorance mencoder model money number print size size size skill score skill tax zeros size | |
Rate this Feed |
Comments (0) |
Feed Details and Statistics |
Updated: 2018-01-27T05:19:55.433-06:00
2012-11-03T09:56:52.415-05:00
But if both forecasters are imperfect, how do we use the data to tell us which forecaster was better? Or how good each one is on a scale with pure guess at one end and a crystal ball at the other end? How can we estimate the degree to which a forecaster was over-confident or under-confident in his own forecasting ability? What about bias? Nick's question had to do with probabilistic forecasts. This problem is this isn't actually covered in any econometrics or probability classes. You have to go to a meteorology department to figure out how to do this without making stupid mistakes.A good example of why good skill scores are needed is the following model. This model was a real model back in the day and actually used by a real weather forecaster. It predicts on every given day that there will not be a tornado in a given town. The forecaster claimed that he would be 98% right, but what we care about is that day when there in fact is a tornado. He got no false positives, but way too many (100%) false negatives. His score was "biased." (This is very different from what econometricians call bias.)The Heidke skill score is a better predictor.Here's a really simple example for yes-no answers:a = Forecast = Yes, Reality = Yes.b = Forecast = Yes, Reality = No.c = Forecast = No, Reality = Yesd = Forecast = No, Reality = YesWe come up with the Heike Skill score by comparing how our model does compared to a coin flip. And how good a perfect model is compared to a coin flip. Then to make the numbers nice we take the ratio of those two results:HSS = (number correct - expected number correct with a coin flip)/(perfect model's number - number correct with a coin flip)This simplifies to:$$HSS = \frac{2(ad - bc)} {(a+c)(c+d) + (a+d)(b+d)}$$ using the above definitions.HSS of one means a perfect forecaster, a zero means the forecaster has no skill, and a negative value says that flipping a coin is actually better than the forecaster.There are many other types of skill scores. They differ based on how they treat rare events, non-events, and systemic vs random errors. You can extend skill scores from a 2x2 table to a larger table for more complex forecasts. This won't do for probabilistic forecasts, however.For probabilistic forecasts, instead of weighing false positives vs false negatives, you are weighing sharpness vs reliability. Here is a skill score for probabilistic forecasts:The Ignorance Skill ScoreLet f be the predicted probability of an event occurring lying on the open interval (0,1). (The ignorance skill score assumes that we are never 100% sure about anything.) Also the ignorance skill score has units of "bits." Yes, it's the same thing we talk about when we speak of "bits" in a computer. It traces its foundations to information theory.And let:$$Ignorance_t(f_t) = -log_2(f_t)$$ when the event happens at time period t,$$Ignorance_t(f_t) = -log_2(1 - f_t)$$ when the event does not happen at time period t, andT = the number of time periods t.The expected ignorance is computed the normal way:$$Ignorance(f)=\frac 1 {T}\sum_t I_t(f_t)$$Standard errors for our estimate of ignorance are also computed the normal way.So, back to your original question, "how do we compare probabalistic forecasters." We can compare the ignorance of the two forecasters by seeing which one is more ignorant.Here is a more intuitive way to understand it:Let's define a function that is "a measure of the information content associated with the outcome of a random variable."Since it's a measure of information, then it should have the following properties:1) This measure of event $$A_i$$ happening depends only on the probability, $$p_i$$ of $$A_i$$ happening.2) It's a strictly decreasing function of $$p_i$$. This is so that the higher the probability, the less useful our prediction of event $$A_i$$.3) It's a continuous function of $$p_i$$. We only want infantesimal changes in probability to cause infantesimal changes in information.4) If an event A_i is the intersec[...]2011-09-05T02:59:03.488-05:00
2011-07-07T13:32:16.232-05:00
A directed graph of patent citations of the top 40 cited organizations (by unique patent assignee number) from 1980 to 2006.(image) |
2011-06-09T12:06:03.322-05:00
2011-02-17T17:05:42.662-06:00
Over the last month, the price of bitcoins has increased from roughly 40 cents to where now a bitcoin is now worth more than a dollar on the Mt. Gox exchange.2011-02-12T23:50:13.066-06:00
Bruce Bartlett says the government is using an axe to cut the budget instead of a scalpel. Well, apparently with a 1.5 trillion dollar deficit out of a 4 trillion dollar budget, we need a chainsaw... an axe is simply too small.2011-02-12T23:46:27.234-06:00
Greg Mankiw misses the point, it isn't a non-independent central bank that Ron Paul wants. It is independent money issuing banks that are not central.2010-07-18T07:44:30.559-05:00
Here is my little sandpile model. I am trying to teach myself a bit about self organized criticality, so I coded this up.Well, here is the code!Note: I edited this post because I found a small bug in the original code.#! /Library/Frameworks/Python.framework/Versions/Current/bin/python# I ripped off the mencoder stuff from thee matplotlib website# That stuff is copyright Josh Lifton 2004, used with permission# The rest is copyright J. Emrys Landivar 2010## 'Permission is hereby granted to use and abuse this document# so long as proper attribution is given.'import numpy as npimport matplotlib.cm as cmimport matplotlib.mlab as mlabimport matplotlib.pyplot as pltimport matplotlib.colorbar as cbarfrom numpy.random import randnfrom scipy import integrateimport matplotlibmatplotlib.use('Agg')import subprocess # For issuing commands to the OS.import osimport sys # For determining the Python version.table_size=31 #the size of the table that the sandpile is onz_crit=4 #the critical size of a stack of grains, it will fall if there are more grains that thatnum_grains=1000 #how many grains to dropz=np.zeros((table_size+2,table_size+2)) #thisis the sandpile and the table it rests onadd_location=(table_size/2+1,table_size/2+1) # sets the middle of the table to drop grains on#print 'Executing on', os.uname()print 'Python version', sys.versionprint 'matplotlib version', matplotlib.__version__not_found_msg = """The mencoder command was not found;mencoder is used by this script to make an avi file from a set of pngs.""" #this was shamelessly ripped off from elsewheretry: subprocess.check_call(['mencoder'])except subprocess.CalledProcessError: print "mencoder command was found" pass # mencoder is found, but returns non-zero exit as expected # This is a quick and dirty check; it leaves some spurious output # for the user to puzzle over.except OSError: print not_found_msg sys.exit("quitting\n")z2=z.copy()fig=plt.figure()for g in xrange(num_grains): z[add_location[0],add_location[1]] += 1 #drop a grain for x in xrange(1,table_size+1): for y in xrange(1,table_size+1): if z2[x,y] > z_crit: #check for colapse z[x,y]-=4 #colapse z[x+1,y]+=1 z[x-1,y]+=1 z[x,y+1]+=1 z[x,y-1]+=1 z2=z.copy() #drop the grains over the edge off the table. z[0,0:table_size+2]=np.zeros(table_size+2) z[0:table_size+2,table_size+1]=np.zeros(table_size+2) z[0:table_size+2,0]=np.zeros(table_size+2) z[table_size+1,0:table_size+2]=np.zeros(table_size+2) #plotting!! ax=fig.add_subplot(111) ax.set_title('Height of the Sandpile') cax = ax.imshow(z, interpolation='nearest') cax.set_clim(vmin=0, vmax=8) cbar = fig.colorbar(cax, ticks=[0,3, 5, 8], orientation='vertical') filename = str('%03d' % g) + '.png' plt.savefig(filename, dpi=100) print 'Wrote file', filename plt.clf()#print sum(sum(z))#make the moviecommand = ('mencoder', 'mf://*.png', '-mf', 'type=png:w=800:h=600:fps=25', '-ovc', 'lavc', '-lavcopts', 'vcodec=mpeg4', '-oac', 'copy', '-o', 'output.avi')#os.spawnvp(os.P_WAIT, 'mencoder', command)print "\n\nabout to execute:\n%s\n\n" % ' '.join(command)subprocess.check_call(command)print "\n\n The movie was written to 'output.avi'"print "\n\n You may want to delete *.png now.\n\n"[...]2010-07-03T07:05:26.863-05:00
A new NY times story about how manufacturing jobs are having difficulty hiring people because of a skills shortage.2010-05-16T07:07:20.472-05:00
This is what people who are economically alarmed believe.2010-04-27T23:21:00.061-05:00
"Adam Smith told us that capitalist would always collude to get government to make it tough for competitors to enter the market place; the current financial reform bill does this in spades with big casino. Of course Sarbanes Oxley already did that; but this make the cheese more binding. If the goal is to lock in big combines and outfits too bid to fail, and be sure no one will compete with them and take some of their market share, the current "reform" bill is a very good continuation of the work started by Sarbanes Oxley."
2009-12-09T14:34:00.603-06:00
Could a few hedge funds decide to begin issuing debt denominated in the hedge fund's own debt? If the funds became known for stability and their debt was highly rated, it would then function as a free bank.2009-12-07T12:09:23.861-06:00
There are really two big schools of modern economic though, the bottom-up and the top-down. The bottom up school is made up of the Austrians and the New Institutional Economists. The top-down school is made of the Monetarists and the new Keynesians.2009-12-04T02:41:11.000-06:00
We had Keynsianism which was based on fiscal policy. We have Monetarism which was based on monetary policy. Will we have a new macro paradigm which is based on legal and institutional policy?
Doc Merlin, I did my dissertation on currency hoarding and the underground economy. The ratio of the tax rate to the interest rate is the number of years you can hoard income in cash before you would have been better off paying taxes. In the US that ratio correlates with the cash/GDP ratio over time. That ratio is high in Japan for two reasons; high taxes and low interest. So hoarding cash to evade taxes is profitable. Ironically, the liquidity trap may make their underground economy bigger.
2009-12-04T00:36:21.256-06:00
Hernando De Soto observes referring to the informal sector, "Whether you are inside the bell jar or outside, you will be taxed. What determines whether you remain outside is the relative cost of being legal."2009-11-05T00:22:29.143-06:00
I just started reading the book, "The Mystery of Capital" by Hernando de Soto, and it occurred to me that one of the functions of private property was to mostly concentrate risk and reward to the persons who were responsible for the action. It seems obvious now that I think about it.2009-11-05T00:04:29.780-06:00
I have to further define 'Expectations.' An expectation is something that an economic actor acts upon based on perception of the future. For example, a doomsday cult believes that the there is a UFO in the tail of the comet Hale-Bop. This UFO will take their spirits away to a better place when they die if they die while Hale-Bop is near earth. They then rationally proceed to commit mass suicide. They weren't insane or irrational, just horribly wrong.2009-11-04T17:44:58.577-06:00
I need a new word for "model" when referring to the ideas in the economic actors' heads that cause them to act certain ways. "Model" causes to much confusion because it seems to refer to more formal and academic types of thinking. Maybe "expectations" is better?Economic actors have expectations, when they act on their expectations and they turn out to be correct, these expectations are rewarded. When they are wrong they are punished.
2009-11-04T16:27:02.250-06:00
This is in response to Scott Sumner's post on his blog, which I recommend reading.2009-10-25T05:33:27.798-05:00
Ok, at first glance, we have a lot of random people. They all have a model in their head of something they want to do.2009-10-16T03:23:38.474-05:00
Type your TeX math expression, in $$$$...$$$$.2009-10-16T01:42:21.483-05:00