Interview with Toni Falvo
Interview conducted in 1995.
"I think the most challenging problem is knowing how to use statistical data and 'gut' feelings in making station decisions."
Toni Falvo holds the position of Research Director at the ABC affiliate in Chicago - WLS-TV. She has worked in the broadcast industry for fourteen years and during her career has made numerous contributions to the growth of WLS - now the city's most watched station.
Her role as the station's chief researcher and data analyst experience at a major ABC affiliate lends an insider's perspective on the varied uses of statistics and decision-making in the television industry. Decision-making and revenue generation within the broadcast industry are heavily dependent on gathering and correctly interpreting data in many cases within very short time periods. Ms. Falvo's role in this process at the nerve center of the station demands both statistical analysis and intuition.
Aczel: Please tell us something about your background in statistics.
Falvo: I took the required basic business statistics course, but nothing beyond that.
Aczel: In what ways is decision-making at the station dependent on statistical data?
Falvo: As a television station, we are evaluated by our ratings performance and position in the market. From Sales to News, decisions can be based on what "the numbers" are telling us. In Sales, how we price and sell our product is based on the strength of our ratings performance and our competitor's. In News, we may make changes and adjustments to content and talent based on those same ratings. In Programming, ratings and audience composition play a key role in decisions.
Aczel: What are the primary sources of your data? Is it generated by your department, externally, etc?
Falvo: The A.C. Nielsen company is our source for ratings data. We also use various qualitative sources, such as Scarborough Research (market specific consumer, media and retail information). We periodically conduct our own research studies via phone surveys or focus group studies.
Aczel: How do you use and interpret Nielsen data?
Falvo: Since Chicago is metered market we receive "overnight" ratings every day. We can immediately see how well a program is doing. We track programs daily and conduct month-to-month and year-to-year comparisons of regularly scheduled programs. We look for positive stories and trends to develop sales pieces to sell our station. We can also see how well our news or local programming is doing, or if there is a competitive concern. When you are looking at a particular program you want to see if it maintains its audience throughout the show, any audience drop-off may signal a problem with content or talent. How well a show maintains its lead-in or increases audience from its lead-in is another way to interpret Nielsen information. If you see a drop-off in your audience and a gain in your competitors we can conduct a study to see where our audience is going minute-by-minute to see if our competitor is gaining audience at our expense. There is also a variety of Nielsen studies to analyze ratings by household income, education, employment characteristics to help define your audience composition.
Aczel: How are the Nielsen viewers selected? How are they motivated?
Falvo: Stations subscribe to the Nielsen service, but we do not have anything to do with the selection process. There are very strict and specific rules regarding the Nielsen sample that must be in place to guarantee the integrity of the sample.
Nielsen follows a very specific selection process for the metered sample. Their goal is to select and maintain a representative sample of television households. They use census data and geographical data to select and recruit sample households. Once a household is selected all televisions are metered and they may remain in the sample for up to five years. There is turnover of the sample every month due to planned and unplanned turnover. You have people moving, wanting out of the survey and other reasons.
People are more willing to be a part of the metered sample than the diary sample because of the amount of work involved. With a meter there is virtually nothing the home has to do but watch TV. With a diary the home must write in what they watch and when. All Nielsen sample participants are given a small compensation.
Aczel: What types of statistical research, polling, etc., does the station generate or sponsor?
Falvo: We will conduct, at least once a year a very detailed research study. We work in conjunction with a vendor that we have selected, to conduct either a telephone survey or a focus group survey. In fact, this year we tried a new technique with our focus groups. The focus group participants were given a hand-held device that (enabled the) to key in positive and negative responses to the video (being tested). This proved to be helpful at the time of the focus group. People seemed more focused and precise with their comments since they had a visual reference and recorded their own responses. It also gave us information to analyze after the groups. All the respondents were recorded individually and we were able to analyze the data based on different variables.
Aczel: Do you use any inferential statistics in your work? If so, what and how?
Falvo: We project audience ratings performance for all programming on the station. Audience projections are based on past performances of a particular program to project it's future ratings performance, or if a show has not run before, you would look for a similar show to base your projection on. You are also looking at the time of day a show is running and what the HUTS (households using television) are during that time period. You may also look at the competitive situation to see what available audience you can expect to (obtain).
Aczel: Do you use statistics in evaluating economic data?
Falvo: We use economic data in two ways. First, in a sales application we would analyze the economic status of a household that watches a particular program. That information becomes a sales tool. For example, if you have "X" amount of households with an income of $75,000+ and they watch your station at a particular time more than any other station this becomes a way to sell your station.
We also use economic data in analyzing the Nielsen sample. Based on U.S. Census statistics we want to make sure that the Nielsen sample accurately reflects the economic composition of the market.
Aczel: Do you design public opinion polls in a statistical way? How do you interpret and report the findings?
Falvo: As I mentioned earlier, we do at least one major study per year. We may do a similar study each year so we can track results year-to-year. Or we may have a specific objective and design a study around that.
We do not conduct public opinion polls, our studies are based on a questionnaire of specific questions that we conduct over the phone. We decide what base we will need, and try to model our recruitment process to mirror the Nielsen sample. Which means we want to get the right ethnic and demographic mix so the sample is now skewed. We randomly select homes in the geographic area we are targeting and select the people by age and sex that we want included. A telephone survey should not last more than 20 minutes and should be a mix of close-ended (selection of responses) and open-ended, where you would encourage individual responses (which is more time consuming).
Once you have collected all the data you can make a variety of assumptions based on the age and sex of your viewers and what the main purpose of the survey was. If you are trying to find out what your viewers feel about your station or a competitor's you can get a clear picture based on a survey that has an adequate base of people and is set up correctly.
Depending on what the study was on, all the departments that this study was (conducted) for would meet together and discuss the results. Many time decisions are influenced by what is (discovered) in these studies, but never solely based on these results.
Aczel: From your perspective, what are the most challenging, statistically-related problems the station deals with?
Falvo: I think the most challenging problem is knowing how to use statistical data and "gut" feelings in making station decisions. I think there is a skill or a sense that you get, especially after being in the television industry for a period of time, that you know when something will work. [After you look at the facts, the ratings, the demographics, etc. you also have to have a "gut" feeling that this is the right decision you should make. I don't think you can have one without the other. Statistical facts alone and "gut" feelings alone will not work.]
Aczel: Given that advertising rates are essentially governed by the Nielsen ratings, does the station or network ever attempt to "verify" their accuracy by conducting your own surveys? If so, have any statistically relevant discrepancies surfaced? What happens in those cases?
Falvo: There is the EMRC (Electronic Media Rating Council) that accredits the Nielsen service. They are the "watchdog" of the ratings service. If problems are uncovered they are brought to Nielsen's attention.
Aczel: At your station, how would you statistically determine the saturation point of certain types of programming, e.g. local news, game shows, talk shows, etc?
Falvo: Statistically there is no real measure of "saturation." Your measure is your viewers, once they stop watching you know you need to make a change. And you will see that (loss) if you are trending and comparing your programming month-to-month and year-to-year. Also you see trends in particular types of programs. For example, at one time there were numerous game shows in daytime TV, those game shows have now been replaced with talk shows and the cycle goes on. In primetime you see the same cycles, police shows, medical shows and variety shows, they start with a big audience appetite for a particular type of show and after there are enough of them their appeal starts to wane.
