Zillow Zestimate has been the go-to source for consumers to check what their house is worth for many years now. Zillow leverages the traffic that this brings to their web site to sell other real estate services (not valuation products!). The Zestimate has been amazing at driving traffic to the Zillow web site, but does it really provide an accurate value estimate for most properties?
Editor’s Note: We our happy to have another guest post from Bruce Hahn, CCIM, CRE, MAI, SRA. Bruce is a fee appraiser living in Nevada. He was George Dell’s guest in our new series of webinars. Bruce and George were speaking about How to Avoid a Reconsideration of Value (ROV). Watch the Replay Here.
Zillow has a large staff of quantitative analysts and data scientists that have developed their algorithm, which is a “Black Box.” That is, the methodology in that algorithm is proprietary and is not fully revealed, so you really cannot understand how they calculate the provided value in the Zestimate. In Zillow’s case that is probably okay, because people don’t generally make financial decisions based on a Zestimate. Until Zillow did!
Zillow has been in the news a lot during 2021. Of particular note, was their house flipping program, Zillow Offers, which leveraged the Zestimate algorithm to forecast what a home would be worth several months out. The intention was to make a cash offer, make minor updates and resell the house within a few months for about a 2% profit. Zillow would make most revenue from the additional services they provide – like title insurance and other transaction related fees.
What went wrong? Quite a few things actually. The supply chain created problems obtaining needed materials for renovations, delaying some resales. The algorithm had a hard time accurately estimating the very rapid price increases during the first half of 2021 that slowed suddenly during the second half of the year. Furthermore, buyer preferences shifted during this period and the algorithm had a very hard time incorporating these changing tastes. Zillow also faced competition for homes from other competitors for the flipping of homes and in some instances paid more than the algorithm suggested to make acquisitions.
Zillow Offers ultimately overpaid on many homes as a result of these factors. Although they made a slight profit with this operation in the second quarter of 2021, they faced a loss of about 6% during the third quarter and shut down the Zillow Offers program during November. During the fourth quarter of 2021 Zillow has laid off 25% of its staff and the company’s stock has lost about two thirds of its value since the first quarter of the year. Ouch!
So what went wrong for Zillow? I believe the answer is fairly simple. Data science is a mix of statistics, mathematics, and computer power. But most important, and overriding these three elements, is the need for a domain expertise. I do not believe that Zillow has domain experts (real estate appraisers) that are involved in their algorithms. Some elements of the market are not easily measured – like buyer sentiment – which can change quickly. Such changes are difficult for machine learning to pick up. Unless someone with domain expertise can identify changes and help modify the algorithm model to accurately reflect the changes, the output reliability will suffer.
So why will data science likely work better for appraisers than for Zillow? Because experienced appraisers bring significant domain expertise with traditional appraisal methodology. They also possess information and knowledge of the market that cannot be captured by an algorithm alone. It takes domain expertise to make a good model from data and then fine tune it at appropriate times when changes in market conditions warrant.
Have you mixed data science skills into your appraisal practice to augment your traditional appraisal methodology skill set?
November 24, 2021 @ 4:12 am
Academics for Machine Learning all emphasize the importance of Domain Expertise lack of it can lead to failure. Appraisers are Domain experts in Real Property Valuation, Appraisers are Data Scientists but do not use any of the applicable data science tools. Do not let this Zillow fiasco lull you into believing these types of tools will not work or go away. This is the learning experience that can open the door for appraisers to step in and use data science tech to take back what has been taken
The real question is do we as a profession have the energy to do it considering the average of the membership. I got my Master Degree in data science at 58 years old. If I can do it so can you. This is the future like it or not. Embrace and control it, don’t let it control us.
November 24, 2021 @ 9:01 am
Bruce misses the boat here. While his theories have some validity what really sunk Zillow is the sellers outsmarting Zillow data science. Zillow’s estimates work around averages but did “average” sellers really take the offers? Many or most sellers knew thier homes needed significant repair or updating to bring them up to the value Zillow was willing to offer. Of course they jumped at the chance to cash in! The result is that Zillow got stuck with a big pile of “Lemons”.
Zillow claims a shortage of labor and materials. This is partial truth. The whole statement should read more like “We paid too much and could not find enough cheap labor and materials that would have helped fix our mistake”. There’s plenty of labor and materials but their market rate fees were more than Zillow could afford to pay given their high acquisition cost. Basic economics.
Had an actual appraiser and/or home inspector been inspecting each of these properties this problem could have been recognized in a glance. Something Zillow’s data science missed.
November 24, 2021 @ 1:44 pm
Why not have a ” field inspector ” or appraiser do random checks. This provides more validity for the local appraiser.
November 24, 2021 @ 7:05 pm
Good read. I was wondering why some of the most recent valuations I did on properties that were owned by Zillow were so out of whack with the market data and market trends.