<p>Before Numbeo was created (April 2009), no other free database about a cost of living (with structured data and indices) existed. <p>Other cost of living reports had the data behind their research hidden or expensive to purchase. Their research was very limited in the number of cities included. It is difficult to scale up without a significant increase in expenses since they relied on manually collected data. Also, there were no insight about the error rate in their manually collected data. Manual collection of cost of living data is error prone: <ul> <li>there is a different price during the year - price oscillation (i.e. cheaper fruits and vegetables during the summer; or high fluctuation of potato price because of lack of storage and high moisture)</li> <li>in different supermarkets, bars and restaurants prices of items are usually different</li> <li>there are different types of milk, cheese, etc. with different prices even in the same supermarket</li> <li>the country could face temporary shortages of a given item which could drive the price temporary up (i.e. rice shortages)</li> <li>if only one person collects the price, possibility of human error is higher</li> </ul> <p>Reports available before the year 2009 usually include just an index, which is not enough for a personal estimate since a person is not an <i>average</i> person due to different lifestyles such as: <ul> <li>the size of a family (number of dependent persons)</li> <li>dining out or eating at home</li> <li>renting or owning an apartment</li> <li>driving or using a public transport</li> <li>drinking alcoholic drinks and smoking or not</li> </ul> <p>Other available cost of living sources didn't provide a systematic way to extract custom indices. Numbeo provides a world-class software for extracting various economic indicators for free (i.e. using our "Basket of goods and services" tool). <p>Before the Great Recession (World Economic Crysis of 2007-2009) price of properties worldwide tended to look like a crazy to the founder of this website. The price of a small apartment in a third world country he currently lives in was same as 310 ultra modern TFT monitors at that time. The wild speculation in property prices suggested that people really needed a tool for a speculation or to turn their speculation down. <p> So, that's how Numbeo was born. Numbeo: <ul> <li>provides free information about prices</li> <li>allows a person to estimate their own expenses</li> <li>uses the wisdom of the crowd to get as reliable data as possible</li> <li>provides a system for various systematic research on our big dataset</li> </ul> <p> <p> <h2>Methodology</h2> <h3>Collecting and processing data</h3> <p>To collect data Numbeo relies on user inputs and manually collected data from authoritative sources (websites of supermarkets, taxi company websites, governmental institutions, newspaper articles, other surveys, etc.). Manually collected data from established sources are reentered twice per year. <p>We perform automatic and semi-automatic filters to filter out noise data. The simplest filter is working as follows: if, for a particular price in a city, values are 5, 6, 20 and 4 in a recent time span, the value 20 is discarded as a noise (as it value is more than 4 times than the average value) <p>Another filter discards ¼ (one quarter) of lowest and highest inputs as borderline cases have a higher probability to be incorrect. Out of remaining entries, the lowest, highest and mean value are calculated and displayed. <p> There are more sophisticated filters in use. The filters are performing better when there are more inputs. <p>One of the advanced filters tries to eliminate bad training data. It digs into discarded data (spam data) and if notices irregularities, it moves them back into the calculation. <p>To summarize our filters, Numbeo uses heuristic technology to get the data quality. Using the existing data Numbeo periodically discards data which most likely are incorrect statistically. <p>Numbeo also archives the values of old data (our default data deprecation policy is 12 months, although we use data up to 18 months old when we don't have fresh data and indicators suggest that inflation is low in a particular country). The values of old data are preserved to be used for historical purposes. <h3>Aggregating data for a country</h3> To aggregate data for a country, we use all entries (for all cities) to calculate country average data. Note that it is different from the aggregating calculated data for all cities in that country (for which we have data in the database). Due to underlying internal formulas used (discarding top and bottom 25% of the data before calculating display values), sometimes low and high price of an item in one city might look not on par with low and high values of that country. That anomaly appears due to underlying formulas used. So, in calculations for the country, we are weighting a city by the number of contributors. Since they are the higher number of inputs for a country than for a city, data showed on a country level, in general, contains lower noise than data showed on a city level. <h3>Currencies</h3> We do use multiple currency feeds including European Central Bank feed to update our internal currency exchange rates almost every hour. For each entry of the contributors, we do save in our database the value in EUR, USD and currency of the input (using current exchange rate). When calculating averages, we do reuse one of those entries based on currency stability and predominant currency in the country to try to minimize cross currency comparison errors. <p/>To show historical data, we do use monthly historical exchange rates to calculate data (mid-month currency exchange rate). If end users choose a custom display currency for displaying historical data in a year, the mid-year currency exchange rate is used to calculate displayed data. <h3>Taxes</h3> Our data about prices shall have GST and VAT included. Our average salary data shall contain the value after income taxes. So we can use these data directly to estimate local purchases power. <h3>Calculating indices</h3> Cost of Living Index is built based on our <i>best guess</i> of average expenses in a given city for a four-person family. Weights are subject to change over time. But since the methodology is not hidden, as the moment of writing these weights are as follows:
Antes de la creación de Numbeo (abril de 2009), no existían otras bases de datos gratuitas sobre coste de vida con datos e índices estructurados.
Existían otros informes sobre coste de vida, pero su metodología era secreta o bien se pedía mucho dinero para poder acceder a ella. Su ámbito era muy limitado en el número de ciudades incluidas. Era difícil aumentar la escala de los estudios sin incurrir en gastos significativos, ya que dependían de datos recogidos manualmente. Además, no existían análisis sobre el porcentaje de error en los datos recogidos. Los datos sobre coste de vida recogidos manualmente pueden tender a errores porque:
Los estudios disponibles anteriores al año 2009 por lo general incluyen un solo índice, lo que no es suficiente para establecer una valoración a nivel individual, dado que una persona no supone la persona media, debido a variaciones en el estilo de vida como:
Otros recursos disponibles sobre el coste de vida no proporcionaban una forma sistematizada de extraer índices personalizados. Numbeo ofrece un software global para extraer varios indicadores económicos de forma gratuita (ej. utilizando nuestra herramienta "Bolsa de bienes y servicios" ).
Antes de la Gran Recesión (Crisis Económica Mundial de 2007-2009) los precios de las propiedades de todo el mundo le parecían totalmente descabellados al fundador de esta página. El precio de un apartamento pequeño en un país del tercer mundo en el que actualmente vive costaba lo mismo que 310 monitores TFT, ultramodernos en aquel tiempo. La especulación descontrolada de los precios hacía entender que la gente realmente necesitaba una herramienta para poder controlar o rebajar la especulación inmobiliaria.
Y así fue como nació Numbeo. Numbeo:
Para recopilar sus datos, Numbeo depende de datos introducidos por usuarios y datos recogidos manualmente de fuentes autorizadas (páginas web de supermercados, compañías de taxis, instituciones gubernamentales, artículos de periódico, otros estudios y fuentes, etc.). Los datos recogidos manualmente de estas fuentes autorizadas se renuevan dos veces al año.
Llevamos a cabo filtros automáticos y semi-automáticos para descartar datos incorrectos o “ruido”. El filtro más simple funciona de la siguiente manera: si para un precio determinado en una ciudad, se han introducido recientemente los valores 5, 6, 20 y 4, el valor 20 se descarta como “ruido” (ya que es más de 4 veces el valor medio)
Otro filtro descarta ¼ (un cuarto) de los valores introducidos más bajos y más altos, ya que los casos en los extremos del intervalo tienen una probabilidad más alta de ser incorrectos. De los datos que permanecen, se calculan y muestran el valor más alto, más bajo y promedio.
También utilizamos otros filtros más sofisticados. Estos filtros funcionan mejor cuando se introducen más datos.
Uno de nuestros filtros avanzados trata de eliminar datos de “entrenamiento” erróneos. Mina los datos descartados (datos spam) y si detecta irregularidades los reintroduce de nuevo en el cómputo.
En resumen, Numbeo utiliza tecnología heurística en sus filtros para asegurar la calidad de sus datos. En base a los datos que ya tiene, Numbeo descarta periódicamente los que estadísticamente tienen más probabilidad de ser incorrectos.
Numbeo también archiva los valores de datos antiguos (nuestra política de depreciación de datos por defecto es de 12 meses, aunque utilizamos datos con hasta 18 meses de antigüedad cuando no recibimos datos nuevos y los indicadores sugieren que la inflación es baja en el país). Los valores de los datos antiguos se preservan para utilizarse en registros históricos.
Other cost of living reports had the data behind their research hidden or expensive to purchase. Their research was very limited in the number of cities included. It is difficult to scale up without a significant increase in expenses since they relied on manually collected data. Also, there were no insight about the error rate in their manually collected data. Manual collection of cost of living data is error prone:
Reports available before the year 2009 usually include just an index, which is not enough for a personal estimate since a person is not an average person due to different lifestyles such as:
Other available cost of living sources didn't provide a systematic way to extract custom indices. Numbeo provides a world-class software for extracting various economic indicators for free (i.e. using our "Basket of goods and services" tool).
Before the Great Recession (World Economic Crysis of 2007-2009) price of properties worldwide tended to look like a crazy to the founder of this website. The price of a small apartment in a third world country he currently lives in was same as 310 ultra modern TFT monitors at that time. The wild speculation in property prices suggested that people really needed a tool for a speculation or to turn their speculation down.
So, that's how Numbeo was born. Numbeo:
To collect data Numbeo relies on user inputs and manually collected data from authoritative sources (websites of supermarkets, taxi company websites, governmental institutions, newspaper articles, other surveys, etc.). Manually collected data from established sources are reentered twice per year.
We perform automatic and semi-automatic filters to filter out noise data. The simplest filter is working as follows: if, for a particular price in a city, values are 5, 6, 20 and 4 in a recent time span, the value 20 is discarded as a noise (as it value is more than 4 times than the average value)
Another filter discards ¼ (one quarter) of lowest and highest inputs as borderline cases have a higher probability to be incorrect. Out of remaining entries, the lowest, highest and mean value are calculated and displayed.
There are more sophisticated filters in use. The filters are performing better when there are more inputs.
One of the advanced filters tries to eliminate bad training data. It digs into discarded data (spam data) and if notices irregularities, it moves them back into the calculation.
To summarize our filters, Numbeo uses heuristic technology to get the data quality. Using the existing data Numbeo periodically discards data which most likely are incorrect statistically.
Numbeo also archives the values of old data (our default data deprecation policy is 12 months, although we use data up to 18 months old when we don't have fresh data and indicators suggest that inflation is low in a particular country). The values of old data are preserved to be used for historical purposes.