Update packages automatically

[metacritical]

I think unpkg packages can be updated automatically through a script.

For instance if you go to https://unpkg.com/hls.js . It will automatically redirect to https://unpkg.com/[email protected]/dist/hls.js

which gives us the latest library-version in . the URL, i think this can be used to automatically update
unpkg based packages using a script on CircleCI.

[Deraen]

Retrieving the latest version number is the easy part, with unpkg or without.

With current CI system this could work if we had script that checked compatible packages for version updates, updates build.boot file and commits the changes. This is easy.

I've been reluctant to implement this because the version changes can easily break the packages. If nothing else, packages often add new API calls that should be added to externs. For generated externs we could have task to generate the new externs, but there are some packages (React) where manually written externs are better choice.

Maybe the script could create new branch & PR for each update, and also add the changelog entry for the version to github comment, making it easy to see if the API changed significantly, and if the update requires manual work.

[metacritical]

Thanks i might work on this when i have the time.

[Deraen]

I was inspired to implement two scripts, one finding the updated unpkg packages, and second creating commit and branch with the update automatically:

b2780c5
#1827 example update for React

It will need way to automatically accept changed checksums (using yes | triggers some problem in Boot), and to create Github PR automatically, but otherwise it should be enough to get update PR's for some packages automatically.

[Deraen]

Using changelog in github message will create references to upstream projects, and mention authors, which will cause unreasonable noise for those people, so something needs to be done about that. I guess warpping the changelog in code block would disable references.

[metacritical]

I think this scheme is great for unpkg packages or similar CDN's. But i think one more thing that can be done is design a system which updates packages based on a rules engine for various kinds of lib sources like CDN's, Github etc. For this we might need to maintain some kind of edn log of packages that can be safely updated "atomatically", the CI trigger will only update these packages on its own.

Once we test this scheme works for a particular package and its source without breaking things we will add that package into that log.

Overtime we can keep adding more and more packages to this log which can be safely updated "automatically" from their respective sources without breaking anything downstream. This way we can also cover edge cases of some packages.

What do you think?

[metacritical]

Using changelog in github message will create references to upstream projects, and mention authors, which will cause unreasonable noise for those people, so something needs to be done about that. I guess warpping the changelog in code block would disable references.

I am actually not sure how this works can you explain?

[Deraen]

Yes, I was thinking some form in whitelisting packages where this is known to work. For now I'll test this with React and few other packages. And I don't think this will be completely automatic, these scripts will create the PR's but I'll still go through those before merging them.

I am actually not sure how this works can you explain?

The changelog entry in the react update I tested, linked e.g. to facebook/react#14914 and you can see at the end of the page that there are several references to Cljsjs commits and the PR. And because the changelog also refers the author, they probably got github notice.

[metacritical]

Yes, I was thinking some form in whitelisting packages where this is known to work. For now I'll test this with React and few other packages. And I don't think this will be completely automatic, these scripts will create the PR's but I'll still go through those before merging them.

Thanks for doing this, this will make cljsjs packages extremely awesome and lucrative for a lot of folks who get disappointed when then look at the list and see an old version on the lib not updated for months.

The changelog entry in the react update I tested, linked e.g. to facebook/react#14914 and you can see at the end of the page that there are several references to Cljsjs commits and the PR. And because the changelog also refers the author, they probably got github notice.

Now i understand what you mean, yes this is actually going to be an annoying problem for the authors upstream. How can we remediate it?

[Deraen]

By wrapping changelog text in codeblocks, I think:

@metacritical 
#1827 

I already implemented this c55c1ff

[metacritical]

Aah that makes sense. This is so awesome! Learnt some interesting stuff which i didnt know earlier about github messages.

[metacritical]

This is some exciting stuff, i am gonna tweet about it.