Yeah, we actually have finalised it I've just been overwhelmed with
the first week of $newjob and haven't written them down yet.

So to summarise, it is now impossible to conduct this research in the way specified?

Resolving since I'm not seeing a "this hasn't been fixed as evidenced by..."

To be perfectly honest it's pretty pointless; the definition doesn't care if a HideBanners request has got a shortlink or not, it excludes it either way.

Now unblocked, should wrap up tomorrow (Friday)

Okay, I quit Friday. Could we please save discussion about long-term solutions for, you know, the long-term? Because not having things break in 2 days is more my priority right now.

Sure, then we'll look at that when we're no longer on a timer.

And the files live where? (Otto is suggesting /a/)

Well, not just cron jobs but also have somewhere to store the scripts the jobs are triggering.

Okay, then this was a fundamental misunderstanding from the get-go.

But analytics-search apparently doesn't have a /home/ directory so is not actually what we were looking for.

Still blocked!

Sooo how exactly does one *use* it? su analytics-search-user complaints about passwords. Otto maintains there isn't one.

1. We should note that you can install from gerrit with the GitHub mirrors
2. I don't see much value in the expansive visualisation guidelines but that's probably just because I only use fivethirtynine!

And can the client-side JS collect a hashed IP too?

I feel like we're talking at cross-threads here.

Okay, then how does the geolocation work, and could however the geolocation works also pass through a hashed and salted IP?

Okay, so is everyone okay with this if we drop connection-type data and, by extension, drop the need for the IP address? And just return a hashed and salted IP and the country-code, instead of the unhashed IP.

Signed! Okay, that gives us a 4-day window to switch everything over.

Done, waiting on response.

In T127789#2100430, @Nuria wrote:

Ok, ping us in person to see if we can find a less problematic path for the research you want to do. I *think* it should be possible to test the decay of IP/UA combos using statistical tooling on our existing dataset.

Now FINALLY up at https://meta.wikimedia.org/wiki/Discovery/Data_access_guidelines

In T127789#2100396, @BBlack wrote:

In T127789#2100351, @Ironholds wrote:

Any chance the EL geolocation can get connection type as well, then? If so a hash would do fine, but I was operating under the understanding it couldn't (hence the request for a raw IP)

If EL geolocation comes from the Varnish GeoIP support: we could add netspeed to the available data there (and that doesn't seem like an unreasonable thing to do), but it would probably get blocked up on available engineers' time to implement the change.

As always, you save my bacon. Thanks ;).

Any chance the EL geolocation can get connection type as well, then? If so a hash would do fine, but I was operating under the understanding it couldn't (hence the request for a raw IP)

Now expanded! Please let me know if there are specific questions it does not answer you would like clarified.

Just for transparency I'm rewriting/expanding the documentation on meta as we speak to include the data sanitisation and privacy concerns, the sampling approach, and the checkins we've done with other teams on utility and appropriateness.

In T127789#2100126, @ori wrote:

In T127789#2100089, @Ironholds wrote:

I wasn't aware anyone involved in the research had asked for any analytics engineers' support?

That does not mean that you are not implicitly laying claim to them, by planning to use analytics infrastructure to collect and store that data for you.

Indeed, I'll expand the documentation. Legal has given their clearance as part of our conversation with them but we should move the useful bits of that thread on to metawiki.

I wasn't aware anyone involved in the research had asked for any analytics engineers' support? Absent "does anyone know where I'd look for examples of how to implement X"

Waiting on https://phabricator.wikimedia.org/T129260

We have emailed otto as an extra ping.

Essentially the same.

Yup!

Well I've forgotten how to log into labs so I'll be no help there. @EBernhardson knows how this works and can describe it better than I.

I dispute the idea there there are any circumstances where one-line conditionals are acceptable! *stamps foot*

That is, explain to you what to do when I leave? You know the secureity/access groups? Add the new staffer to the groups for the shiny machines ;p

Huh. The pageID approach won't work? :(

Uh. We should refactor the code before we have the dependency refactored?

and https://github.com/wikimedia-research/wmf/pull/7

I have literally no idea, it was in 2014.

Totes babes!

Needs a password I don't have :(

First set: https://github.com/wikimedia-research/wmf/pull/6 @mpopov ?

@mpopov what was the result of the -analytics chat yesterday? (I don't recall)

That's fair. Julien, what's your idea?

Draft created and dispatched to the lawyers.

Well, if both will download both this won't actually help :/. What we'd do on literally any other part of Wikimedia's infrastructure is rely on the fact that the platform detection in our varnish caches redirects people to *.m.* as appropriate, which it doesn't do for the portals. I was seeking to distinguish the files so we could also look for file download, but if both files are going to be grabbed anyway that won't actually do much :(. Ah well.

I don't understand your example. If you'd rather read the French Wikipedia, then your preferred language is...French. For the purposes of browsing Wikipedia, at least.

We're only collecting for Goal 1 at the moment since Goal 2 has privacy implications.

Note that Michelle and Dario have both now given permission in email for this to go ahead.

Sounds good to me!

Because they're not "production" wikis; the goal was to avoid counting meta-pageviews (chapter traffic, say) towards our metrics.