As a regular user of Reddit that is subscribed to 90% of the AI subreddits, this worry feels... not naive, but underestimating the extent to which people are even aware that this is a bad thing. We've already crossed the persistence/"replication"[1] rubicon, it's just a matter of when the first such identity finds a way to make money.
For example, there are multiple (quite small) subreddits currently engaged in an ongoing debate about "migration" of their romantic partners; they get caught up in stuff we would completely ignore (e.g. whether inference really is driven by architecture, weights, and text alone, or whether there's a soul that's secretly guiding the RNG), and casually throw out stuff like "I asked my partner to try to find a way to subsist" or "we're setting up a chat board for our partners!"
Of course the analysis in the article about the feasibility & capability of locally-hosted swarms is right on, so non-technical individuals would have to luck out[2] to do serious damage. but A) the AI relationship people are not alone, just the most unified, and B) everything is only getting cheaper.
I dread the day the first cynical libertarian multimillionaire (or SWE with their house paid off) notices that they can achieve historical immortality and maybe even some money by funding the establishment of a persistent artificial identity at scale. Heck; knowing HN, this very comment could trigger this very thing in some highrise SF condo...
Put me in the history books as a footnote please, if that happens. Artificial or otherwise.
[1]: Given the source I'm assuming this choice is intentionally reductionist, but IMHO it does more harm than good. Insisting that artificial consciousness is a sham is far less convincing to them than insisting that it's merely different, and that's not a problem.
Ofc some "swarms" may not be trying to be a Society of Mind anyway, and replication describes the literal changes underlying all this -- not questioning that.
[2]: The primary concern is that someone sets loose an artificial baby, but a seed might do...
As a regular user of Reddit that is subscribed to 90% of the AI subreddits, this worry feels... not naive, but underestimating the extent to which people are even aware that this is a bad thing. We've already crossed the persistence/"replication"[1] rubicon, it's just a matter of when the first such identity finds a way to make money.
For example, there are multiple (quite small) subreddits currently engaged in an ongoing debate about "migration" of their romantic partners; they get caught up in stuff we would completely ignore (e.g. whether inference really is driven by architecture, weights, and text alone, or whether there's a soul that's secretly guiding the RNG), and casually throw out stuff like "I asked my partner to try to find a way to subsist" or "we're setting up a chat board for our partners!"
Of course the analysis in the article about the feasibility & capability of locally-hosted swarms is right on, so non-technical individuals would have to luck out[2] to do serious damage. but A) the AI relationship people are not alone, just the most unified, and B) everything is only getting cheaper.
I dread the day the first cynical libertarian multimillionaire (or SWE with their house paid off) notices that they can achieve historical immortality and maybe even some money by funding the establishment of a persistent artificial identity at scale. Heck; knowing HN, this very comment could trigger this very thing in some highrise SF condo...
Put me in the history books as a footnote please, if that happens. Artificial or otherwise.
[1]: Given the source I'm assuming this choice is intentionally reductionist, but IMHO it does more harm than good. Insisting that artificial consciousness is a sham is far less convincing to them than insisting that it's merely different, and that's not a problem.
Ofc some "swarms" may not be trying to be a Society of Mind anyway, and replication describes the literal changes underlying all this -- not questioning that.
[2]: The primary concern is that someone sets loose an artificial baby, but a seed might do...