Some instances on the Fediverse offer full-text search of statuses (not just self-authored ones), and I'm wondering how to deal with that. I'm tempted to block every instance doing so, because the current expectation of our users is that their statuses will not be full-text searchable by strangers.

Opinions? What should we do with instances offering full-text search for all statuses?

@rixx Why is that the expectation of users?

I'd assume that anything I post on the internet will be able to be found via some search engine, does Mastodon provide some protection against that?


No protection, in fact the feature is built into mastodon, its just up to the admin if they want to turn it on locally or not.


@freemo Could you elaborate where the feature of full-text search for all statuses is a part of Mastodon? The documentation states that their FTS integration supports only logged-in users searching for statuses from their own posts, mentions, and favs.

That's noticeably differnt from being able to FTS all users' statuses, hence the consideration.

@rixx Sure. All you do is put the following line in your mastodon's .env file and it turns on full-text searching (assuming you also are running the proper services): "ES_ENABLED=true"

Here is the official documentation that explains how to turn off or on this feature:

@freemo That is the exact part of the documentation I linked, so I'm not sure why you point to this. It states very explicitly that "Mastodon’s full-text search allows logged in users to find results from their own toots, their favourites, and their mentions. It deliberately does not allow searching for arbitrary strings in the entire database."

@rixx ahh i see what you mean now.. notice you can also search for mentions.. .I think this explains the confusion. When i search i **do** get other peoples statuses but only if they mention me (I see that now).

What server do you know of offers full text search of all statuses? As far as i know if a instance allows full text search it is this feature they are enabling... are there any servers that have some hack to this feature?

@freemo Since your instance states that it offers FTS, and you have repeated this claim as including all statuses, I had to assume (for the sake of our users) that you were doing what you said you were doing.

@rixx When did I publicly claim it included all statuses? I just double checked our about page and all it says is i enabled the FTS feature... I dont think I ever publicly made the claim it included other posts until a few minutes ago, w3hich was an error on my part.

Also to be fair what could have also avoided it is if even a **single** person had just asked me about it.

@freemo You stated to me, just a couple of minutes ago, that you did FTS, erroneously, so I'm correcting the assumption.

@rixx ahh yes, that was in error a few minutes ago. But that is also the first tie I made that claim publicly (as I was mistaken as to what FTS feature did exactly)...

Im more concerned with your statement that the recent drama around qoto was my fault for stating something I never said publicly (aside from a few minutes ago where I clearly made a mistake).

@freemo I did *not* say that "the recent drama around qoto was your fault". Assigning blame does not further the conversation. Pretending others assigned blame doesn't help, either.

@rixx I'll try a bit harder in the future to be aware of your sensabilities. Just let me know if i ever make you uncomfortable and ill try harder to avoid those behaviors. Sorry again, very unintentional that I upset you.

@rixx To be clear this is the exact wording on our about page. As you can see we dont state anywhere that our searches do not honor the privacy as you suggested: "Full text searches - usually you can only search hashtags and usernames"

I would hope most level headed people would see that and understand it is refering to enabled the standrd FTS feature since we never made any public statements to the contrary.

@rixx Where do you see it say in that text that it only works for your own posts?

We enable full text searched on QOTO, we do so using the guide you linked (and I linked) above. We do nothing special beyond that.

But I can clearly search through posts other than my own (I just tried), but I am limited to search only for posts I have permission to see, and yes I do need to be logged in.

So seems you just have the feature mistaken. It does, in fact, allow you to search posts of other users.

@freemo Please refer to the documentation you linked. If you are only using this feature, it's misrepresented in how it's portrayed by and about your instance, and most of the people protesting your instance could have been avoided by a clearer statement.

@rixx Can we get this feature (searching one's own toots) for
I'm scrolling my own page about once a week searching for something I posted two months ago and it would be much nicer if I could just search for it.

@ytvwld I have downloaded my archive and grepped over it in the past. It works and is probably faster than manually searching all my posts

@christianbundy You can opt out of search-engine indexing, and our Terms state that our users' statuses must not be systematically scraped.

Not everything that is technically possible is ethical.


The search full-text search feature in mastdon already acts ethically in this way as fa as i know. It does **not** scrape posts that indicate they dont want to be indexed. You can only search for texts or posts that a server has given permission to see.


@strangeglyph @christianbundy The indexing opt-out is a per-user option, and also not quite the point in question.

@rixx @christianbundy Ah, I misunderstood, sorry. I do agree with the others in this thread that blocking every instance permitting full-text search may be unrealistic. Maybe collecting a list of what / how many instances would be blocked first would be helpful?

Also on a related note, what happens when I follow people and their instance gets blocked. Do they get removed silently or are they still on my follow list, I just do not receive updates from them anymore?


> Not everything that is technically possible is ethical.

Of course, I'm just concerned that it's technically possible. It feels *very* dangerous to set an expectation that can't be implemented through technical means. Could you confirm that everyone knows that others can do full-text search?

Sorry, it's just that yesterday I learned about and I'm trying to get a handle on which security measures are real and which ones are everyone following the rules.

@christianbundy "Could you confirm that everyone knows that others can do full-text search?"

This is where this conversation stops, because "could you please make statements about everybody else's knowledge" is not a tenable position to shove onto somebody.


> This is where this conversation stops

I don't understand the hostility, but okay.

I was asking whether the opt-in setting explained the reality of the situation or whether it gave users a false expectation. If it's the latter, I think we should fix it.

@christianbundy This was not intended as hostility, just to let you know that this escalation of to discussion scope is not something I'm willing to follow at this time.

@rixx Do whats right for you guys, but you know this is a feature that is built-in to mastodon (though can be toggled on and off)... Its also the default on non-mastodon servers. So you will have to block the majority of the fediverse.

Moreover, how will you even compile a list of servers that do this. Most dont announce it since its such a standard feature.

@rixx What if someone hosts a fake instance to make toots searchable? Is there any way to have certainty about the search features of every Instance which is federating with

And there also exists tools which use the API for full-text-searching of statuses.

@rixx And yes, I understand the problem that users of get the daily experience that full-text-searches are impossible and this alters their behavior and the content of their toots.

@eest9 The fact that people can work around restrictions does not make the workaround ethical, and does not mean that we have to support this kind of behaviour. "Peple do it, so fuckit, don't bother" is not an attitude I'm willing to take.

@rixx @eest9 Good to think of these things as features not as problems. Maybe we need to change the documents rather than the code as we are talking about this on a "open" network. Which should have been made clear on sign up to your instance?

@rixx also worth considering (but maybe that's what you meant):
whether to admin block instances that actively seek federation in order to scrape your instance's posts.

Our users voted against admin block and for user chosen actions.

@fihu That would be a nice feature for Mastodon: Admin suggested blocks.

You can see what blocks the admin suggests and either manually or automatically add and remove them from your block list.
The safe default could be "use the admin blocklist" and users who do not want to block these users/instances can change this to manual import, so they decide themself.

This would defederate us from too many instances.

And I consider privacy of public toots as non-existent anyway. Too many instances with admins that I don't know get them, either directly or via boosts.

@allo @rixx yep its a trust based open network. Why are we talking about figleaf privacy in a open network. Think this is a #geekproblem what do you think?

@rixx Is it possible to prevent a client to generate a full text search index of the global timeline?

If not, what's the difference if a server does it vs. a client individually?

@rixx My expectation would be to have full text search on the (recent?) toots from accounts I'm following or individually favored toots. Anything more seems too much.

@rixx in my opinion blocking such instances only creates a false sense of security. In the end everything is searchable anyhow. Its just slightly less convenient.

@rixx Seriously, the whole discussion is ridiculous.

I totally agree with @KopfKrieg and @allo: If something is posted publicly, it's exctly this: public. Hiding itmfrom FTS would only convey a false sense of non-existing privacy.

I'm somewhat shocked about the naive encounter to social networks from people i thought of knowing better.

@rixx @ordnung if it is a public toot then I don't really see a problem. More so if the user does not opt out of search engine indexing.

@rixx @ordnung if it's public, it's public. Someone might as well just scrape the profile and then perform the search on that.

Sign in to participate in the conversation – a Fediverse instance for & by the Chaos community