IDEAS home Printed from https://ideas.repec.org/a/spr/astaws/v18y2024i2d10.1007_s11943-024-00340-6.html
   My bibliography  Save this article

Flat rent price prediction in Berlin with web scraping

Author

Listed:
  • Camilo Meyberg

    (Freie Universität Berlin)

  • Ulrich Rendtel

    (Freie Universität Berlin)

  • Holger Leerhoff

    (Amt für Statistik Berlin-Brandenburg)

Abstract

Internet data pose a challenge to the traditional system of official statistics, which relies on more conventional sources such as surveys and registers, not readily adaptable to rapid changes. Expanding this system to include internet data is currently at an experimental stage, exploring these sources’ potentials and benefits. This paper describes a project conducted within the ESSnet Trusted Smart Statistics – Web Intelligence Network framework. It investigates the use of online apartment listings to analyze the rental market. We used web scraping to extract information from two online real estate portals for flats in the city of Berlin. Using this data, we developed a model to predict rental prices per square meter based on the accommodation’s features and location within the city. We detected offers which appear in both portals by means of statistical matching and removed duplicate offers. Missing values were treated by multiple imputation. The prediction model is a semi-parametric approach where the postal districts are used to describe the location effect. Comparisons with microcensus results and the local rent index reveal significant differences between the market of online flat offers and the stock of existing flat contracts. Interested readers will find the commented programming code in the internet supplement.

Suggested Citation

  • Camilo Meyberg & Ulrich Rendtel & Holger Leerhoff, 2024. "Flat rent price prediction in Berlin with web scraping," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 18(2), pages 245-278, June.
    • Handle: RePEc:spr:astaws:v:18:y:2024:i:2:d:10.1007_s11943-024-00340-6
    DOI: 10.1007/s11943-024-00340-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11943-024-00340-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11943-024-00340-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Adam Czerniak & Michał Rubaszek, 2018. "The Size of the Rental Market and Housing Market Fluctuations," Open Economies Review, Springer, vol. 29(2), pages 261-281, April.
    2. Edward E. Leamer, 2007. "Housing is the business cycle," Proceedings - Economic Policy Symposium - Jackson Hole, Federal Reserve Bank of Kansas City, pages 149-233.
    3. Michal Rubaszek & Margarita Rubio, 2020. "Does the rental housing market stabilize the economy? A micro and macro perspective," Empirical Economics, Springer, vol. 59(1), pages 233-257, July.
    4. Rebecca C. Steorts & Timo Schmid & Nikos Tzavidis, 2020. "Smoothing and Benchmarking for Small Area Estimation," International Statistical Review, International Statistical Institute, vol. 88(3), pages 580-598, December.
    5. Heij, Christiaan & de Boer, Paul & Franses, Philip Hans & Kloek, Teun & van Dijk, Herman K., 2004. "Econometric Methods with Applications in Business and Economics," OUP Catalogue, Oxford University Press, number 9780199268016.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michal Rubaszek & Margarita Rubio, 2020. "Does the rental housing market stabilize the economy? A micro and macro perspective," Empirical Economics, Springer, vol. 59(1), pages 233-257, July.
    2. Adam Czerniak & Michał Rubaszek, 2018. "The Size of the Rental Market and Housing Market Fluctuations," Open Economies Review, Springer, vol. 29(2), pages 261-281, April.
    3. Mateusz Tomal, 2022. "Testing for overall and cluster convergence of housing rents using robust methodology: evidence from Polish provincial capitals," Empirical Economics, Springer, vol. 62(4), pages 2023-2055, April.
    4. Michał Hebdzyński, 2024. "Price-setting factors or revealed preferences? How to understand the results of hedonic models and hedonic indices of the housing rental market that base on listings data?," Bank i Kredyt, Narodowy Bank Polski, vol. 55(4), pages 519-550.
    5. Navarro-García, Juan Carlos & Madrid-Guijarro, Antonia, 2016. "Real economic activity and accounting information in Spanish construction and real estate firms," Revista de Contabilidad - Spanish Accounting Review, Elsevier, vol. 19(1), pages 21-30.
    6. Michael T. Kiley & Jean-Philippe Laforte & Rochelle M. Edge, 2008. "The Sources of Fluctuations in Residential Investment: A View from a Policy-Oriented DSGE Model of the U.S. Economic," 2008 Meeting Papers 990, Society for Economic Dynamics.
    7. Hertrich Markus, 2019. "A Novel Housing Price Misalignment Indicator for Germany," German Economic Review, De Gruyter, vol. 20(4), pages 759-794, December.
    8. Seitz, Franz & Baumann, Ursel & Albuquerque, Bruno, 2015. "The information content of money and credit for US activity," Working Paper Series 1803, European Central Bank.
    9. Matthew Rognlie & Andrei Shleifer & Alp Simsek, 2018. "Investment Hangover and the Great Recession," American Economic Journal: Macroeconomics, American Economic Association, vol. 10(2), pages 113-153, April.
    10. Goodness C. Aye & Stephen M. Miller & Rangan Gupta & Mehmet Balcilar, 2016. "Forecasting US real private residential fixed investment using a large number of predictors," Empirical Economics, Springer, vol. 51(4), pages 1557-1580, December.
    11. Hassan Belkacem Ghassan & Abdelkrim Ahmed Guendouz, 2019. "Panel modeling of z-score: evidence from Islamic and conventional Saudi banks," International Journal of Islamic and Middle Eastern Finance and Management, Emerald Group Publishing Limited, vol. 12(3), pages 448-468, July.
    12. Mehmet Balcilar & Rangan Gupta & Stephen M. Miller, 2015. "The out-of-sample forecasting performance of nonlinear models of regional housing prices in the US," Applied Economics, Taylor & Francis Journals, vol. 47(22), pages 2259-2277, May.
    13. Francesco Furlanetto & Francesco Ravazzolo & Samad Sarferaz, 2019. "Identification of Financial Factors in Economic Fluctuations," The Economic Journal, Royal Economic Society, vol. 129(617), pages 311-337.
    14. Jakub Borowski & Adam Czerniak & Beáta Farkas, 2023. "Diverse Models of Capitalism and Synchronization of Business Cycles," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 65(4), pages 681-712, December.
    15. Kouretas, Georgios P. & Pawłowska, Małgorzata, 2020. "Does change in the market structure have any impact on different types of bank loans in the EU?," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 65(C).
    16. Carlos Garriga & Aaron Hedlund, 2019. "Crises in the Housing Market: Causes, Consequences, and Policy Lessons," Working Papers 2019-33, Federal Reserve Bank of St. Louis.
    17. Nyakabawo, Wendy & Miller, Stephen M. & Balcilar, Mehmet & Das, Sonali & Gupta, Rangan, 2015. "Temporal causality between house prices and output in the US: A bootstrap rolling-window approach," The North American Journal of Economics and Finance, Elsevier, vol. 33(C), pages 55-73.
    18. Joshua Aizenman & Brian Pinto, 2013. "Managing Financial Integration and Capital Mobility—Policy Lessons from the Past Two Decades," Review of International Economics, Wiley Blackwell, vol. 21(4), pages 636-653, September.
    19. Lars Osberg, 2014. "What’s So Bad About More Inequality?," Working Papers daleconwp2014-01, Dalhousie University, Department of Economics.
    20. Finn E. Kydland & Peter Rupert & Roman Sustek, 2012. "Housing Dynamics over the Business Cycle," NBER Working Papers 18432, National Bureau of Economic Research, Inc.

    More about this item

    Keywords

    Web scraping; Flat offers; Official statistics; Rent indices; Statistical matching; Multiple imputation; Semi-parametric regression;
    All these keywords.

    JEL classification:

    • R21 - Urban, Rural, Regional, Real Estate, and Transportation Economics - - Household Analysis - - - Housing Demand
    • R31 - Urban, Rural, Regional, Real Estate, and Transportation Economics - - Real Estate Markets, Spatial Production Analysis, and Firm Location - - - Housing Supply and Markets
    • L86 - Industrial Organization - - Industry Studies: Services - - - Information and Internet Services; Computer Software

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:astaws:v:18:y:2024:i:2:d:10.1007_s11943-024-00340-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.
    pFad - Phonifier reborn

    Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

    Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy