package-rewiew-sig-2008-10-23

--- bpepple has changed the topic to: Package Review SIG Meeting
bpeppleHi, is anyone around for the Package Review sig meeting?
kushalbpepple, me me
limburgherpresent.
kushalbpepple, let me wake up chacha_chaudhry :p
bpepplekushal: good, I was a little worried I would be the only one here. ;)
kushalhehe
limburghernah.  Just had to finish lunch ;)
bpeppleok, let's wait another minute or so to see who else shows up before starting.
limburgher\me taps foot awkwardly.
* chacha_chaudhry wakes up finally
bpeppleok, let's get started.
let's do a quick roll-call.
BrianPepple
limburgherJon Ciesla
chacha_chaudhryRakesh Pandit
bpeppleSince this is our first meeting, what would you guys like to discuss first?
limburgherways to get people to do reviews?
I'd also like to see attention paid to Merge Reviews.
kushalKushal Das
ubertibbsI'm here.
bpepplelimburgher: I would also, though I think our first priority should be making the New package reviews manageable first.
limburgherubertibbs: glad you could make it.
ubertibbsMerge reviews are a "do when there's nothing else to do" thing for me.
And so far there's plenty of other reviews around, so...
kushalbpepple, for new reviews , review-o-matic is coming up :)
ubertibbsUnfortunately review-o-matic doesn't actually review packages.
limburgherubertibbs: I agree, but I feel like it's silly to have some of the things so far out of compliance 3+ releases after the merger.
bpepplekushal: yeah, I saw that.  How about in a little bit we talk about that.
kushalbpepple, main idea is simple
kushaleverytime someone put a new review request in bz
RoM (review-o-matic) will do an automagic review and post the result as comments
ubertibbsPlease stop calling what it does a "review".
kushalwhich is only to help the reviewer
ubertibbsIt doesn't do reviews; it just runs some checks.
kushalubertibbs, which is review , isn't it ?
limburgherr-o-m is a great concept.  I just think more focus should be on the backlog.
rdieterpre-review-omatic?
kushalrdieter, :)
limburgherreview>checks.
ubertibbskushal: No.
I mean, it's not going to do anything that people shouldn't already be doing before submitting packages.
rdieterreview-helper?  review-tool?
kushalubertibbs, it is going to check the spec according to guidelines
limburgherr-o-m should really just be rpmlint on steroids.  The packages still need the wetware sanity check, sniff test, etc.
kushaland md5 checks
bpeppleok, let's take a step back for a moment.  One of the things I like to see us do is to produce a report that shows some weekly stats on the package review queue.
limburgheri.e. how can a script decide what 'legible' means?
kushaland rpmlint on scratch builds
limburgherbpepple: +1
kushalchacha_chaudhry, is working on spec file review work
ubertibbsbpepple: Sure, stats are good.
limburgherstats can drive incentives, goals, etc.
chacha_chaudhrypoint is get as much as possible checks into rpmlint and all other fedora specific checks which can be automated should go into review-o-matic.
bpeppleAnd publish them weekly, to keep in mind just what the backlog is.  Hopefully, that will help drum up some support for reviews.
ubertibbsIncentives?
bpeppleubertibbs: I've been giving that some thought also.
limburgherubertibbs: Nothing concrete in mind, just throwing it out there.
https://fedoraproject.org/wiki/PackageMaintainers/PackageStatus is great.
ubertibbsYes, that's Chris Iseli's work.
It was very useful in the old days, but the volume is so high now that it needs some tuning.
limburgherMaybe a "Reviewer of the Month" in FWN.
ubertibbs: How is volume affecting it?
bpepplelimburgher: Yeah, I was thinking about modifying c4chris script, since I believe it needs some work for some of the recent bugzilla changes.
ubertibbsWell, some of the useful stats are pushed so far down the list because of people like, well, me, who have done tons of reviews.
limburgherbpepple: OIC.
bpeppleWe could provide some Fedora swag drawing for folks that do a certain number of reviews in a month.
ubertibbsYou can't even see the new folks in the stats.
limburgherubertibbs:  Good point.  It's good to see the uber^H^H^H^Hsuper-reviewers, but also give the
ubertibbs: newbies something where they can say, "look, I made a dent in it".
bpeppleubertibbs: does c4chris script show a weekly total on reviews, or was it totals?
ubertibbsI think it just did an overall total.
rdietera per week or monthly total would be interesting to see, could form a good basis for recognizing current work.
bpepplerdieter: agreed.
limburgherrdieter: +1.  Reviews are very important, but somewhat thankless.
bpeppleDoes anyone want to work on modifying c4chris script to meet our needs better?
I might have some free time to help out on that.
ubertibbsNot for myself, but if we're going to reward work we should go back and look at the folks who did a pile of work in the past.
limburgherbpepple: Maybe, though my -fu may be tested. . .
limburgherubertibbs: Sort of a lifetime acheivement thing?  Little gold statue. . .call it the Tibbsy. . .
:)
ubertibbsI haven't looked at that script in ages; it is still in Perl or was it ported to something else.
?
bpeppleubertibbs: I thought it was in python, but I could be wrong since I haven't looked at it in ages.
limburgher$_DIETY, by perl's rusty.  Hope it's python.
s/by/my/
kushal:)
bpeppleI'll contact c4chris and grab the latest copy.
ubertibbsIt should be in CVS.  The module is status-report-scripts.
limburgherDoes it produce html, or barf out wikicode?  If option B, does it work with mediawiki or is it trac-specific?
bpeppleubertibbs: thanks, I'll give it a look.
ubertibbsOne other thing to note is that we can't even track who does reviews reliably.
limburgherReliably meaning 'often' or 'of high quality'?
ubertibbs"can't reliably track"
chacha_chaudhrybpepple: I can help with perl/python script.
bpeppleubertibbs: We can track who give the approval flag correct? Just not every one that provided comments.
limburgherGotcha.  Wouldn't that be a BZ query though?
ubertibbsYou can track who set the flag, or who the ticket is assigned to.
bpepplechacha_chaudhry: thanks.
limburgherRight, no need to catch practice reviews.
ubertibbsI can fix up badly-set flags but then it shows me as setting the flag.
And at least one reviewer insists on assigning the ticket back to the reporter once the flag is set just to get the ticket off of his BZ homepage.
So if we're going to rely on process for generating reliable stats, we need everyone to actually follow the process.
limburgherubertibbs: Eew, that's annoying.
ubertibbs: Bit of a horse-to-water issue, but I see no choice.
ubertibbsThere's also the issue of doing triage and how to credit folks that do it.
limburgherSo, once we have stats, which we now seem to have a plan for, what will we do with them?
ubertibbsPost them and talk about them, I guess.
bpepplelimburgher: Publish them reliably.
limburgherubertibbs: Do people triage reviews, or just breakage bugs?
bpeppleThat will at least raise it's profile a bit.
ubertibbsWell, I've tried to interest the triage folks in review tickets as well.
They're a bit swamped and we don't really have any process guidelines to give them in any case.
bpeppleubertibbs: yeah, that seems understandable.
limburgherubertibbs: How would that work?  With breakage, you make sure it's against the right package.
ubertibbsPlus the automated tool folks are working on could do a lot of this.
limburgher: Well, you make sure the package submissions actually build, for one.
You'll also notice Mamoru and I doing a pile of gardening on the queue.
limburgherubertibbs: Good point.
ubertibbs"Did you still want to get this in?"  "There's been no response for N weeks; are you still there?" and such.
limburgherAlso, making sure the SRPMS and SPEC aren't 404.  Like, if they're in koji and got pruned. . .
ubertibbs: I have seen that. And quite happy about it.
ubertibbsWell, cleaning out the dead tickets sure helps to get the count down.
limburgherTrue dat.
ubertibbsI really need to redo the http://fedoraproject.org/PackageReviewStatus stuff, but at least it hides blocked tickets and such now.
limburgherubertibbs: Which is helpful.
ubertibbsYes, it hides all of the mingw32 stiff for now.
Between that and globus, there are probably another 250 packages in the queue.
bpeppleIs anyone reviewing the mingw stuff?
ubertibbsWell, I looked at the -filesystem package; I'll probably finish that off soon.
Everything is blocked on that one, so...
bpeppleok, does anyone have any other brain-storming ideas on how to increase the number of reviewers?
limburgherbpepple: Don't judge it by the movie, but what about a sort of Pay It Forward policy?
ubertibbsI hate to suggest it, but things hinge on people reviewing at least a percentage of what they submit.
limburgherYou do my review, you link another review for me to do while you're working on mine.
Of someone else's.
ubertibbsWell, I rarely submit packages but I do plenty of reviews.
bpeppleyeah, we've always encouraged that, but maybe we could advertise it better than we have in the past.
limburgherubertibbs: Me too.  a big chunk of what I maintain are orphans.
rdieterubertibbs: right, the usual answers... cross/trade reviews, new sponsoree's encouraged to help with other reviews, etc...
ubertibbsMaybe prioritize the queue by the ratio of reviews done / package submissions ?
rdieterI like that!
ubertibbsThat would be tough (both tough to do and tough on the submitters, though).
bpepplethat's an interesting idea, probably tough to implement though.
limburgherubertibbs: Explain.  Details.  Intriged.
[sic]
ubertibbsI can provide neither explanation  nor details.
It was an idea.
limburgherLike, you submit a lot, but review next to never, we would like to see X reviews by you before we touch yours?
rdieterkinda sorta
ubertibbsWell, not "before we touch yours", but they'd go at the end of the list or something.
limburgherSet up like a leader board graded by that ratio, so people looking for reviews to do will
ubertibbsBut nobody says that anyone has to do reviews in any specific order.
And honestly, some people just want to maintain packages. Is that actually a bad thing?
limburghersee the packages for those whove reviewed first.
ubertibbsReviewing requires a rather different skill set.
limburgherNo requirements, just those who review get bumped up the list and higher visibility.
* ianweller sucks at reviewing, btw ;)
rdieteranother possible area of focus would be engaging (other) SIGs, for example, us kde-sig'ers usually try to keep an eye out for and help on kde-related reviews.
ubertibbsYes, and for that reason I tend to stay away from them and concentrate on something else.
Right now my focus is on ancient NEEDSPONSOR tickets.
limburghersame with games SIG
ubertibbsTo me, the old needsponsor tickets are our worst embarrassment, because we're basically turning someone away from Fedora.
rdieterubertibbs: totally +1
limburgherubertibbs: +1, that's why I became a sponsor.
bpeppleubertibbs: agreed.  with the new containment that was implemented it should hopefully make it easier to sponsor someone.
ubertibbsIdeally we'd address every review ticket, but especially the needsponsor ones, within some time period.
ubertibbsUnfortunately our process is complicated and people often don't read it and so don't indicate the needsponsor stuff on their own tickets.
That's another triage issue.
bpeppleDo we want to try to have coordinated review days?  In the past we haven't had much success with them, but it might be worth trying again.
rdietersomething where some sort of priority queue could help, needsponsors get a bonus, nearing some timeout would get a bump too.
limburgherbpepple and rdieter: +1  each
ubertibbsI can sort the PackageReviewStatus pages by anything, I guess.
rdieterbpepple: worth a shot, I promise to participate this time. :)
bpeppleyeah, I'll actually attend this time also. ;)
ubertibbsHonestly I think some reviews would go better if we could team up on them.
bpeppleThat what I was thinking also.
limburgherThat usually works well.
chacha_chaudhryubertibbs: need for sponsor triage could be automated also I guess.
rdieternod
ubertibbsBecause otherwise we're going to have to assign the tough tickets to various people.  "You've been designated to take one for the team."
chacha_chaudhry: If there's a definite link between FAS and bugzilla addresses, yes. Otherwise, no.
chacha_chaudhryubertibbs: yes
limburgherMaybe if we triaged reviews into groups by potential comps category.
sorted on the stats list.
limburghernot in the bz
bpeppleOk, we've brain-stormed a bunch of ideas.  How about I write-up some minutes which lists them, and then we work on looking at implementing some of them.
ubertibbsSure.
limburgherchacha:link how?
bp: Works for me.
bpeppleDo we want to have weekly meetings? Or bi-weekly? Or some other timeframe?
ubertibbsOr use a mailing list?
I find IRC to be generally inefficient, and some people can't attend things at the same time.
bpeppleI'm fine with that, though I'm leery of starting another mailing list.
limburghermail is better for me, for tibbs' reasons.
rdieterjust use fedora-devel ?
ubertibbsI was going to suggest just using fedora-packaging.
But I don't know what's more appropriate. I read both, so....
(for various definitions of "read")
rdieterworks too.
limburgherrdieter: And have our own threads, and maybe attract more members.  -devel has wider readership, i think.
bpeppleubertibbs: fedora-packaging has less traffic, but devel reaches a larger audience.
limburgherWe could agree on a subject tag convention, so I could procmail out this topic. :)
[ReviewSIG]
bpeppleOk, so does anyone object to using the devel list?
limburgherbp: No.
ubertibbsNot me.
chacha_chaudhryno
rdieterno
bpepplealright, we'll use the devel list then.
chacha_chaudhry ChanServ
ubertibbsAt least until we get flamed for it.
limburgherFlames?  In -devel?  Naaah. .
rdieterthat's the sprit.  you're not doing it right, unless you get at least a flame or 2.
limburgherWhaddy mean I'm not doing it right?  <napalm>
bpepplerdieter: ;)
ubertibbsWould we like to discuss specific review tickets, like the bugzappers do?
Or maybe considering holding some kind of IRC workshop for people who want to learn how to review packages?
bpepplemight be worthwhile.
limburgherMaybe a "5 oldest" List?  It might get repetiive, but it'll clean aout the cruft.
Maybe 3 normal, 2 Merge each time?
ubertibbs: +50
ubertibbsmerge reviews need a ton of triage before we can do much with them.
limburgherTriage how?  Like set reporter to owner?
ubertibbsWell, you can't do that unfortunately.
limburgherClean out removed packaged?
ubertibbsBut you can at least CC them; many of the owners have changed since the tickets were opened.
limburgherubertibbs: That's been a huge headache of mine.
ubertibbsYes.  just /msg zodbot and keep typing queries.
bpeppleAlright, we're coming up on the hour.
limburgherCool. So are we adjourned?  Sadly, I have a 2pm.
bpeppleSo, chacha_chaudhry, limburgher, and I will work on a script to produce a weekly report on package review stats.
And I'll write up a summary on the mailing list of the various ideas we've come up with, and we'll work on implementing some of them.
anyone have anything else to add? Or should we wrap up?
ubertibbsI'm happy to brainstorm with someone about how we might teach reviewing to others.
I tried it in person (without any previous organization) and failed miserably.
bpeppleubertibbs: It might be worthwhile to have a review trac at FUDCon.
Assuming you will be there.
ubertibbsI wouldn't be against it but don't know if I'll be going to future fudcons.
rdieter:(
limburgherDepends on where they are.
ubertibbs$4 million budget deficit in my college.
bpeppleouch.
ubertibbsThe US fudcons are generally in Boston or Raleigh.
limburgherMy employer will pay for 2 volunteer days and travel, but not sure if FUDCon counts.
Any chance of one in Chicago? :)
rdietersomeday maybe, I'd like that too.
ubertibbsdgilmore was trying to get one in Chicago.
rdieterFUDCon @ dgilmore's house!
bpeppleyeah, it would be closer for me, but having a convertion in Chicago is pretty pricey.
limburgherI'll bring beer.
bpepples/convertion/convention/
limburgherJust use iconv like everyone else. . .
rdietera good venue could be hard to come by.  BU/boston has it's advantages
limburgherJust do it on Grant Park.  On the grass.  Picnic!
bpepplerdieter: yeah, I taked to stickster_afk about it a bit when he was in Ohio.  A good facility is one of the biggest hurdles.

Generated by irclog2html.py 2.5 by Marius Gedminas - find it at mg.pov.lt!