It seems to me that the whole article is about proving this point.
Relevant quotes:
> The secrecy of the algorithms further pushes people toward conformity. If you are worried that the US government will classify you as a potential terrorist, you're less likely to friend Muslims on Facebook.
> If you know that your Sesame Credit score is partly based on your not buying "subversive" products or being friends with dissidents, you're more likely to overcompensate by not buying anything but the most innocuous books or corresponding with the most boring people.
> Uber is an example of how this works. Passengers rate drivers and drivers rate passengers; both risk getting booted out of the system if their rankings get too low.
> Many have documented a chilling effect among American Muslims, with them avoiding certain discussion topics lest they be taken the wrong way.
Those are trying to establish a loss from having these algorithms. They don't show whether the benefits of open source outweigh the costs. That part only seems to be handwaved in the sentence I quoted.
I think you're being unfair. The author has given us many clear issues with closed source algorithm that would not exist with open source algorithms.
Showing that the benefits outweigh the costs would require a model of the impact the (closed or open) algorithmic judgments on the society, and thus a model of the society itself.
Such model would inevitably be subject to even more subjectivity and discussions.
If you're not buying it, it's your choice, and you may even be right. But that doesn't make the whole argument unreceivable. Instead, you should mention the benefits of closed source algorithms for the society, they're not that obvious to me.
That's a definite statement, and shouldn't be made without such a model. If you only have handwavey models, you can't claim one is "much worse" than the others. The level of confidence isn't justified. I'm fine with a list of possible bad outcomes, but don't claim way A is much worse than way B without stronger evidence.
>Instead, you should mention the benefits of closed source algorithms for the society, they're not that obvious to me.
One is mentioned in the part I quoted: that they're harder to circumvent. This isn't the same as regular security software where they theoretically are secure, but practically have holes, and so there's gains by opening it up and letting people report holes. There would be no theoretical security model here, it would only be probabilistic. There may not be a "patch" for the kind of holes (I'll give an example soon). So security by obscurity might make more sense here than in general.
Example: suppose our data shows that people that drive car X have a greater security risk. If this is kept secret, we are able to more effectively target people (probably, many different factors will combine until someone is an extreme risk, but this is simplified.) The fact that X is weak evidence of risk is information that becomes useless relatively soon after it is publicized (because potential terrorists just put "don't buy X" on their checklists.) So publicizing the algorithm would concretely lead to worse results.
Realistically, car model probably isn't too predictive, but possibly the union of all behaviors that are easy to change when known predicts risk reasonably well.
Relevant quotes:
> The secrecy of the algorithms further pushes people toward conformity. If you are worried that the US government will classify you as a potential terrorist, you're less likely to friend Muslims on Facebook.
> If you know that your Sesame Credit score is partly based on your not buying "subversive" products or being friends with dissidents, you're more likely to overcompensate by not buying anything but the most innocuous books or corresponding with the most boring people.
> Uber is an example of how this works. Passengers rate drivers and drivers rate passengers; both risk getting booted out of the system if their rankings get too low.
> Many have documented a chilling effect among American Muslims, with them avoiding certain discussion topics lest they be taken the wrong way.