what’s my responsibility towards the web, anyway?
A lot of the responsibility of skewed survey reports/sampling biases comes down to the survey/report maintainers, I believe. Like when the flawed report is observed the first time, why is it not analysed and digged into for reasons leading to the ‘not-so-true’ numbers.
Why is the activity not focused on the next time to ensure the sampling bias (if that’s what the cause is) is reduced. This is so important for the resource maintainers to look at because it impacts the developers’, users’, and the community’s efforts in general.
When we were looking at the framework usage data for the Web Almanac, the numbers looked clearly skewed due to detection limitations of Wappalyzer.
To avoid confusing users with incomplete/inaccurate measurement of adoption, we ensured to NOT present any wrong information and highlight the detection limitation.
This does justice to the general sentiment that readers get when reading the resource.
