[note: I suspect we mostly agree on the impropriety of open selling and dissemination of this data.  This is a narrow objection to the IMO hyperbolic focus on government assault risks. ]

I'm unhappy with the phrasing of "targeted by the Chinese government", which IMO implies violence or other real-world interventions when the major threats are "adversary use of AI-enabled capabilities in disinformation and influence operations." Thanks for mentioning blackmail - that IS a risk I put in the first category, and presumably becomes more possible with phone location data.  I don't know how much it matters, but there is probably a margin where it does.

I don't disagree that this purchasable data makes advertising much more effective (in fact, I worked at a company based on this for some time).  I only mean to say that "targeting" in the sense of disinformation campaigns is a very different level of threat from "targeting" of individuals for government ops.

I don't have confidence in my models of how coherent and competent governments are at getting and using data like this.  The primary buyers of location data are advertisers and business planners looking for statistical correlations for targeting and decisions.  This is creepy, but not directly comparable to "targeted by the Chinese government".

My competing theories of "targeted by the Chinese government" threats are:

1. they're hyper-competent and have employee/agents at most carriers who will exfiltrate needed data, so stopping the explicit sale just means it's less visible.
2. they're as bureaucratic and confused as everything else, so even if they know where you are, they're unable to really do much with it.  

I think the tension is what does it even mean to be targeted by a government.

Moral weights depend on intensity of conscient experience.

Wow, that seems unlikely.  It seems to me that moral weights depend on emotional distance from the evaluator.  For some, they're able to map intensity of conscious experience to emotional sympathy (up to a point; there are no examples and few people who'll claim that somthing that thinks faster/deeper than them is vastly more important than them). 

Just to focus on the underlying tension, does this differ from noting "all models are wrong, some models are useful"?  

an AI designer from a more competent civilization would use a principled understanding of vision to come up with something much better than what we get by shoveling compute into SGD

How sure are you that there can be a "principled understanding of vision" that leads to perfect modeling, as opposed to just different tradeoffs (of domain, precision, recall, cost, and error cases)?  The human brain is pretty susceptible to adversarial (both generated illusion and evolved camoflage) inputs as well, though they're different enough that the specific failures aren't comparable.

I tend to read most of the high-profile contrarians with a charitable (or perhaps condescending) presumption that they're exaggerating for effect.  They may say something in a forceful tone and imply that it's completely obvious and irrefutable, but that's rhetoric rather than truth.  

In fact, if they're saying "the mainstream and common belief should move some amount toward this idea", I tend to agree with a lot of it (not all - there's a large streak of "contrarian success on some topics causes very strong pressure toward more contrarianism" involved).

Hmm.  I don't doubt that targeted voice-mimicking scams exist (or will soon).  I don't think memorable, reused passwords are likely to work well enough to foil them.  Between forgetting (on the sender or receiver end), claimed ignorance ("Mom,  I'm in jail and really need money, and I'm freaking out!  No, I don't remember what we said the password would be"), and general social hurdles ("that's a weird thing to want"), I don't think it'll catch on.

Instead, I'd look to context-dependent auth (looking for more confidence when the ask is scammer-adjacent), challenge-response (remember our summer in Fiji?), 2FA (let me call the court to provide the bail), or just much more context (5 minutes of casual conversation with a friend or relative is likely hard to really fake, even if the voice is close).

But really, I recommend security mindset and understanding of authorization levels, even if authentication isn't the main worry.  Most friends, even close ones, shouldn't be allowed to ask you to mail $500 in gift cards to a random address, even if they prove they are really themselves.

In deep meditation people become disconnected from reality

Only metaphorically, not really disconnected.  In truth, in deep meditation, the conscious attention is not focused on physical perceptions, but that mind is still contained in and part of the same reality.

This may be the primary crux of my disagreement with the post.  People are part of reality, not just connected to it.  Dualism is false, there is no non-physical part of being.  The thing that has experiences, thoughts, and qualia is a bounded segment of the universe, not a thing separate or separable from it.

Is your mind causally disconnected from the actual universe?  That's the only way I can understand the merging of minds that share some similarities (but are absolutely not identical across universes that aren't themselves identical).  Your forgetting may make two possible minds superficially the same, but they're simply not identical.

I don't know why you think path-based configuration of brain state would be false.  That may not be "identity" for all purposes - there may be purposes for which it doesn't suffice or is too restrictive, but it's probably good for this case.