That may be the New York state total, not New York City.
This page at the NY Times (updated 4/22/2020) reports 10,301 deaths for the City, 14,828 for the State. If the infection fatality rate were 0.12%, that would mean 8.6 million infections in a city with 8.5 million population.
One problem with the New York fatality total is that they are counting victims WITH the virus, not victims OF the virus. Because of that, based on Times data, NYC fatality rate from known cases is 7.4% while the
California rate is 3.8%.
For info, another antibody survey was done by USC
in Los Angeles County resulting in a fatality rate estimate of 0.14% to 0.27%, similar to the Stanford/Santa Clara County estimate:
Based on results of the first round of testing, the research team estimates that approximately 4.1% of the county's adult population has antibody to the virus. Adjusting this estimate for statistical margin of error implies about 2.8% to 5.6% of the county's adult population has antibody to the virus- which translates to approximately 221,000 to 442,000 adults in the county who have had the infection [10 million population]. That estimate is 28 to 55 times higher than the 7,994 confirmed cases of COVID-19 reported to the county by the time of the study in early April. The number of COVID-related deaths in the county has now surpassed 600.
Your point is still a good one: If the fatality rate is that low, infection in New York City must be far more widespread than plausible. However, deaths per million population in NYC (1,200) is so dramatically higher than anywhere else in the US (LA County 60, SF 22), that something weird seems to be going on there.