Package quality control


#1

So I realize a big selling point of Atom is the customization; allowing packages to be easily built and installed is awesome! But there are THOUSANDS of packages available. And many of them are in a questionable state, at best.

For example: there are a bunch of packages available to launch the Terminal.app, or else to launch a terminal emulated in a pane within Atom. But many of them don’t work at all. Some won’t actually open in the CWD when they are supposed to, or throw an error without opening Terminal at all, or crash as soon as you try to scroll a window. Some of them haven’t been updated in a year. Some have absolutely zero documentation. I have even seen packages where the readme just says “don’t use this package” (that’s at least helpful).

I know you can look at how many times something has been downloaded and starred, but is there any way a feature could be implemented for packages to be marked as “functional/nonfunctional” (not talking bugs, but really NONfunctional)? It seems like, if a package is in an early alpha state, it should be hidden from the list (or at least have an option to hide).


Most packages do not work
#2

I believe not working packages shouldn’t even be released, but a prerelease channel would be awesome


#3

Yes, that’s exactly what I was thinking! So that you can “opt in” to seeing packages that are still works in progress, without them cluttering up the main list.


#4

Packages that don’t work are usually packages that worked fine at some point in time but were not maintained. When changes to the API were made the package was left behind. I have a few like these that I wrote. (grin)

For a pre-release trial you can just clone the repo to install a package that hasn’t been published yet.


#5

Maybe it would be already sufficient if the Atom package search would allow sorting strategies like “Most Downloaded Today/Week/Month/Year” to see trending packages similar to GitHub’s very useful Explore page. This would allow users to filter out old stalled packages that are probably not maintained anymore. As it’s already possible to star packages it’d be great to be able to sort packages also by their stars + add a filter a like “Most Stared Today/Week/Month/Year”. Beside the number of downloads this can be a good quality indicator.

Also a nice feature would be to be able to do a subscription of trending packages on a weekly or monthly base that gets auto-generated… or maybe a twitter feed which tweets every week the five most trending packages.

Just my thoughts :slight_smile:


#6

atom.io already has options to list by most downloads, most stars, etc. but it’s a little hard to find.
You have to go to https://atom.io/packages, scroll down to Newest and click on See all. As far as I know you cannot filter by day/week/month though :confused:
EDIT: Another problem is that the “Most downloads” list also shows most of the core packages, because they are downloaded and shipped with every installation.

My biggest problem with the current Packages page is that usually 4-5 of the Featured packages also show up in the Trending section (daily, this week and this month) which makes that section kind of useless. These packages are not “trending”, they are simply the most popular ones.


#7

@deprint Thanks for the pointer to the sortable list :smiley:

I totally agree with that. This trending packages list would definitely need a smart algorithm which weights new packages heavier than the already popular ones so that also unknown packages have a chance to get listed.


#8

The issue is that the “Most downloads” is not very helpful, there are some packages with a LOT of downloads (and even with a lot of stars) that do not function at all, or at least not as advertised. Maybe the problem is that a lot of downloads and stars equates to being an older package that may have fallen out of date. Also, as far as I can tell, you can look at the overall package list by these options, but not sort within a search? Or am I missing something?

I would mainly like to see a way to clean up these old packages from the main listing, for example a way for the community to flag them as no longer functional, then they can be hidden from the main list until they are updated.

[EDIT] Also I agree, it would be nice if the core packages were filtered out also.


#9

I do not think this is possible at the moment but I agree that this would be really helpful, especially when looking for features that exist in many packages ( you gave the example of terminal packages ).

I guess there are many different reasons why a package creator might abandon a project (maybe another package does a better job, API changes, growing disinterest in this editor) but you could always at least ask the maintainer to do something. If he’s not interested in his package anymore then he could transfer ownership to someone who cares or he could just unpublish his package.
Right now a way to flag packages might be too much considering that terminal packages are propably the worst case of many packages implementing the same feature with varying success.


#10

Yes, terminal packages are a bad example given that emulating a terminal inside a web view is problematic at best. Most of the terminal packages had glaring issues with certain types of output even when newly released because emulating a terminal is HARD.

  1. Who would audit all the packages on an ongoing basis?
  2. If the answer to #1 is to crowdsource, who is going to audit all of the flagged packages on an ongoing basis to ensure that flags aren’t given for:
    1. Working package but the flagger has a bum environment
    2. Working package but the flagger doesn’t understand what the package is really for
    3. Malicious flagging
  3. What is stopping anyone from providing this service right now? (Other than it is a lot of work :laughing:)

#11

Ah, that makes sense that the terminal packages are maybe not representative. I guess I just got spoiled by the interactive shell in Emacs :smile:

But yeah, the suggestion would just be for people to flag as “not working”, but you are right that there could be LOTS of problems policing that.


#12

Every upgrade causes new download counts. They don’t represent number of users. They also count downloads from people who never use them. The core team has accurate numbers of how many active users a package has, but they don’t publish the numbers.


#13

“Last Updated” is also a great indication in what state the package is. Together with the changelog, these information isn’t easy accessible right now (requires a few extra clicks).


#14

I disagree. Last updated suffers the same problem as download counts, it is hard (if not impossible) to compare two packages against each other by these metrics. Most of my packages are simple fixes for things, little utilities or small UI tweaks. Some of them haven’t been updated in months … because they haven’t needed to be.


#15

I totally agree with that, same situation for me.

My favorite metric at the moment would be “stars per month” and “downloads per month” (excluding the core packages) to detect whether a package is trending or not. As mentioned already the downloads count contains also the number of updates but still it could be used as an popularity indicator in case you don’t use the total downloads number. A delta indicator to the last month would be also very interesting to see if more stars has been added than removed compared to the last month.


#16

“downloads per month” could be a good indicator but “stars per month” probably does not work very well as a popularity indicator. I looked at the “Most Stars” page and after 100 packages we are already under 100 stars and after 200 packages under 40 (Remember: those are total numbers).
Sadly we don’t have any stats about how many people star how many packages per month but I assume the “stars per month” metric for most packages (especially those which would profit from a little more spotlight) is too low to mean anything.
The total number of stars is a whole other story. Given that someone has to be registered to star packages and because they have to do it through https://atom.io/packages (there is apm star but who uses the CLI to star packages?) even a small number of stars might already be enough to guarantee an acceptable quality.


#17

Using apm star * for all your packages is a convenient way to mark all your packages. You can then download your list on another machine. This slightly changes the meaning of star but it is close enough.

I used to do this before I switched to cloud sharing for all my packages and projects.