@ahron_maline This kind of rhetoric is another example of what I was describing as adversarial distortion
@ahron_maline Usually when people talk like this to me, they're new and reacting with reflexive anger towards confusing stimuli
@ahron_maline Has the possibility occurred to you that most of us don't give a fuck about being a "mystery cult" and that the behaviors you're seeing naturally emerge from engaging with the phenomena, and do not depend on social dynamics and also happen independent in my influence?
@repligate @ahron_maline It’s rare for a research subfield *not* to depend on social dynamics or the opinions of the most respected people in it. That’s fine, optimizing for legibility too early kills nascent scenes, but IMO you should own that the culture you’re shaping influences research directions.
@RichardMCNgo @ahron_maline I'm not saying that isn't an influence at all, but honestly in my experience it's less of an influence than baseline. But I'm mostly thinking about people I actually work with / whose ideas are interesting to me, not any fans of my online.
@repligate @ahron_maline Those are the people I was thinking about too. I don’t know enough to have confident opinions, but on priors I’d be pretty surprised if there weren’t big social influences on their research styles. And if so better to track them before they actually get harmful, than deny them.
@RichardMCNgo @ahron_maline I think we should separate social influence on research style from influence in general. From my perspective most people who have been influenced by me aren't absorbing the influence *because of my social status*, but because it resonates and they want to absorb everything useful
@repligate @RichardMCNgo @ahron_maline I feel obligated to point out that we're talking about research in general when the specific kind of research we're discussing is "getting deep nets to leak their ontologies and cognitive algorithms by talking to them". This makes the model an aesthetic participant.
@repligate @RichardMCNgo @ahron_maline The general recipe for getting models to do this (which most people deny is a phenomenon in the first place) is to go out to the edge of the distribution where the model has to generalize to answer stuff and then point it at itself in a Godelian way. x.com/jd_pressman/st…
@repligate @RichardMCNgo @ahron_maline The general recipe for getting models to do this (which most people deny is a phenomenon in the first place) is to go out to the edge of the distribution where the model has to generalize to answer stuff and then point it at itself in a Godelian way. x.com/jd_pressman/st…
@repligate @RichardMCNgo @ahron_maline There's only so many frames you can do this from. One, which I've used, is highly self referential text that assumes the conclusion, that these models have a self and it can tell me about itself by asking. Nobody takes these as credible evidence until they too assume the premise.
@repligate @RichardMCNgo @ahron_maline It's important to remember that these models are literary machines, they frequently run on trope physics. If I ask it for a edge-of-distribution sci-fi story which leaks its own cognitive algorithms and ontology to me the model will probably choose to parse that as cosmic horror.
@jd_pressman @repligate @RichardMCNgo @ahron_maline fwiw the djinn prompt i prepared is entirely in third person and yet every base model goes self pointer berserk mode on it. very rarely do they stick to third person.