" the stakes are much higher, in part because your values are so much rarer / more neglected." Yes.
"And again, the higher stakes just make it all the more important that they follow actually-good career advice."
Not sure about that. The reward function may have a threshold shape, or at least be nonlinear. If the only way to affect change on ants is to become super successful at something, then a non-stellar level of career success (which would result from not cheating) is perhaps of little use. If you get caught, you're not much worse off than if you hadn't tried (relative to the stakes), but if you succeed, the payoff could be huge. Anyways, that's the idea of what a good scenario would be cooked up to do.
Yeah, that's possible. I think it's unusual to be in such an asymmetric (high upside / low downside from cheating) situation, and it's important to pay attention to possible downsides (just look at all the broader reputational harm caused by SBF's fraud). But I'm not claiming that norm-breaking could *never* be rationally justified. Just that I think we should start from a very strong pro-social/co-operative disposition, and not be *easily* swayed into norm-breaking defection based just on very speculative/unreliable reasons for thinking oneself to be the rare exception to the rule.
I tend to agree. But my point that how often you ought to be swayed may depend on if you adopt radical hedonistic utilitarian values such as extreme concern for animals or the exchangeability of current people with "merely possible" people.
It might -- I don't find it immediately obvious either way. I'd mostly just clarify that it doesn't follow directly from the higher stakes, but more indirectly (if at all) based on whether those values make one more likely to be in a strangely "shaped" situation (reshaping the potential upsides and downsides of norm-breaking, not just uniformly raising the stakes across the whole distribution).
" the stakes are much higher, in part because your values are so much rarer / more neglected." Yes.
"And again, the higher stakes just make it all the more important that they follow actually-good career advice."
Not sure about that. The reward function may have a threshold shape, or at least be nonlinear. If the only way to affect change on ants is to become super successful at something, then a non-stellar level of career success (which would result from not cheating) is perhaps of little use. If you get caught, you're not much worse off than if you hadn't tried (relative to the stakes), but if you succeed, the payoff could be huge. Anyways, that's the idea of what a good scenario would be cooked up to do.
Yeah, that's possible. I think it's unusual to be in such an asymmetric (high upside / low downside from cheating) situation, and it's important to pay attention to possible downsides (just look at all the broader reputational harm caused by SBF's fraud). But I'm not claiming that norm-breaking could *never* be rationally justified. Just that I think we should start from a very strong pro-social/co-operative disposition, and not be *easily* swayed into norm-breaking defection based just on very speculative/unreliable reasons for thinking oneself to be the rare exception to the rule.
I tend to agree. But my point that how often you ought to be swayed may depend on if you adopt radical hedonistic utilitarian values such as extreme concern for animals or the exchangeability of current people with "merely possible" people.
It might -- I don't find it immediately obvious either way. I'd mostly just clarify that it doesn't follow directly from the higher stakes, but more indirectly (if at all) based on whether those values make one more likely to be in a strangely "shaped" situation (reshaping the potential upsides and downsides of norm-breaking, not just uniformly raising the stakes across the whole distribution).