Currently When performing a system upgrade, Yay will first refresh the
database then perform the repo and AUR upgrade. This allows Yay to add
some features such as better batch interaction, showing potential
dependency problems before the upgrade starts and combined menus
showing AUR and repo upgrades together.
There has been discussion that this approach is a bad idea. The main issue
people have is that the separation of the database refresh and the upgrade
could lead to a partial upgrade if Yay fails between the two stages.
Personally I do not like this argument, there are valid reasons to Yay
to fail between these points. For example there may be dependency or
conflict issues during the AUR upgrade. Yay can detect these before any
installing actually starts and exit, just like how pacman will when
there are dependency problems.
If Yay does fail between these points, for the previously mentioned
reasons or even a crash then a simple refresh will not cause a
partial upgrade by itself. It is then the user's responsibility
to either resolve these issues or instead perform an upgrade using
pacman directly.
My opinions aside, The discussions on the Arch wiki has reached
a decision, this method is not recommended. So to follow the decided
best practises this behaviour has been disabled by default.
This behaviour can be toggled using the --[no]combinedupgrade flag
It should be noted that Yay's upgrade menu will not show repo packages
unless --combinedupgrade is used.
--ask is no longer used when installing AUR packages, instead pass no
confirm when we know there are no conflicts and wait for manual
confirmation when there are.
This means that when there are no conflicts there should be no change in
behaviour and the user will not need to intervene at all.
The old behaviour can still be used with --useask.
Clean build needs to happen before downloading pkgbuilds so that they
can be deletd before downloading.
Editing and diff viewing needs to happen after downloading the
pkgbuilds.
Prevously we asked to clean and edit at the same time. Then clean,
download pkgbuilds and open the editor.
This poeses a problem for diff viewing and editing. It's likley that the
user will see the diff and use that to decide if they want to edit the
pkgbuild. Using the current method, the user will be asked to view diffs
and edit before actually seeing any diffs.
Instead split cleaning diff showing and editing to three seperate menus
in the following order:
show clean menu
clean
download pkgbuilds
show diff menu
show diffs
show edit menu
edit pkgbuilds
Also each menu is seperatly enableable. By default only the diff menu is
shows. If the user wishes to clean build, edit pkgbuilds or disable
diffs then the user can use the --[no]{clean,diff,edit}menu flags. This
replaces the --[no]showdiffs flags.
The order of targets does somewhat matter. For example doing something
like 'pacman -S db1/foo db2/foo' should cause the second package to be
skipped.
The order of targets also effects in which order they are resolved. This
should make errors more reproducable if any ever occur.
diff viewing can be toggled via --[no]showdiffs. When enabled diffs will
be shown for packages between the current HEAD and upstream's HEAD.
Packages downloaded via tarballs will be shown in full using the editor
git diff is used to show diffs. Therefore the pager for diffs can be
set via the PAGER and GIT_PAGER enviroment variables.
These flags limit operations to only check the repos or only check the
AUR. These flags apply to -S, -Si and -Su.
-a may also be used as a short option for --aur. --repo has no short
option as -r is taken.
-Sc will delete cached AUR data from Yay's build dir according to
the `CleanMethod` specified in the Pacman config file.
-Scc will delete all cached AUR data.
Additionally -Sc will also delete all untracked files in the AUR cache.
This will delete things such as downloaded sources and built packages
but will leave directories behind.
This is very usefull because the only reason a directoiry should exist
in the cache is because of downloaded VCS sources. Non VCS sources are
redownloaded every update because when a new version is released a new
tarball has to be downloaded. But VCS sources are never redownloaded,
updates are simply pulled. For this reason the user probably wants to
keep this data as it is still usefull even after building and installing
a package.
Use git clone over tarballs for pkgbuild downloading during -S. This
option can still be toggled using the config flags.
The config option for selecting clone or tarball will be overiden if an
existing package is cached. The method used to download the package
perviously will be used regardless of the config.
Previously we ran pkgver() right after dowloading sources. This is
a problem because prepare() should be called and all dependencies
should be installed before pkgver().
Instead bump the pkgver while building then get the new pkgver used for
install. Previously we parsed `makepkg --printsrcinfo` to get the new
version. Insead use `makepkg --packagelist` as it is much faster.
Allows searching the RPC for words that may be too short or have
too many results as long as another word in the search will work.
If no words can be used without error then the last error will be
returned and the program will exit.
add22f5957 added error checks to all the
passToPacman commands. This makes `yay -Q nonexistantpackage` return
non 0 as it should. Annoyingly it also made yay print `exit status = n`
which is the error string from passToPacman calls. This error doesn't
add much and is quite annoying, expecially when calling pacman commands
like `-Q` or `-Si` where yay should be kind of 'hidden' and print just
like pacman does.
Now set the error string to "" for pacman commands and don't print an
error if it == "" (avoids empty line printed).
Also behave more like pacman when using `yay -Qu`.
-Qu now has AUR support, it functions identically to `-Pu` and may replace
it in the futre.
Aditionally the pacman options `-n` and `-m` are also supported to
filter out native and non native packages. Other flags are not supported
currently.
Using any other `-Q` will fallback to Pacman.
The callback is set to allways silently say yes, When passing to pacman
for the intall pacman will then ask the question giving the user
a chance to answer.
Similar to the --redownload flag, when specifed targets will be rebuilt
even if an up to date version is cached. --rebuildall can be used to
ensure uninstalled dependencies are rebuilt as well.
Additionally, unlike --redownload there is also --rebuildtree. This
causes a rebuild and reinstall of a package and all of it's dependencies
recursivley. This is designed for when a libary updae, breaks an
installed AUR package due to a partial upgrade. polybar is a common
example
--rebuild allows you to easily skip the cache and rebuild against a newer
libary version. --rebuildtree is a more nuclear option where you can
rebuild the whole dependency tree.
Adds the ability to pick which database to install a package from. This
is extended to also support for AUR packages. For example `extra/git`
and `aur/yay` should both work`. When not explicitly requesting
a database repo packages will be choosen over the AUR.
This features extends to yogurt mode, listings where a package shows up
in multiple database/the AUR is now handled.
The aur does not have a real pacman databse like core, extra ect. But
can be accessed as if was one with `aur/name`. Using Yay with a pacman
repository named "aur" is undefined.
With this The user should never have to manually edit the yay config
file. All options can be set directly through yay using
`yay --<option> <value> --save`
Before setting options such as --topdown would be saved to the config
file automaticly when used. Now this is no longer done by default and
isntead the --save flag must be passed for this to happen.
If --save is passed the config is now saved as soon as the argument
parsing is finished apposed to before where it was saved when yay exits.
This means that config changes will now apply if the user does a ^C
before yay finishes.
If a pkgbuild is already in cache and matches the version on
the aur skip the download.
The version we check comes from the .SRCINFO file on disk which is never
updated. (updates through pkgver() edit the pkgbuild but do not effect
the .SRCINFO). Therefore if the the version of the .SRCINFO matches the
AUR's version there must not be an update.
In the case of the on disk version being newer than the AUR version we
can assume user interaction and they probably do not want it overwitten
so in that case also skip the download.
Use the command `git ls-remote <url> <branch>` to track devel updates
rather than relying on the GitHub API.
This allows devel update to work for every git based source and
elimantes the rate limiting from GitHub.
The yay_vcs.json format has changed to better support packages which
multiple vcs sources and to track the protocols each source uses. And
track the branch that each source tracks in it's fragment.
Add ( and ) to "installed" to match groups and out of date
Show popularity as well as voted in yogurt mode
Show download and install size in yogurt mode
Remove printing of white and black
Fix incorrect message on number menu
Yellow is now almost never used
Use `v` instead of `r` when printing version
show when a page was marked out of date on search and info
Reword a lot of manpage and usage options.
Remove the `-Yg` and `-Y --getpkgbuild` options from documentation and
code as they have moved to `-G` and `--getpgkbuild`.
Update the version in the man page. Because our version is based on our
commit number added a `+` after the version number to indicated that
this is the version the manpage was written at but it also applies to
newer versions.
Added missing `-c --complete` and `-f --fish` options to the manpage.
But left them out of usage.
Added myself to the authors section. I do feel a bit bad about adding
myself and only myself but I thought it was not a good idea to add 20
names and fill up the page. There's currently no rules on who should be
added and not really a huge need for them right now. I guess we can just
add more people when we deem they belong there whatever that criteria
is.
Save the VSC Info as soon as the package install finishes. This should
ensure the VSC db does not end up in an incorrect state if an install
fails or is cancelled by the user.
This also adds better support for split packages. When one or more
packages are installed from the same base each individual package is
added to the db not just the base. This allows us to track individual
updates from the same base so that if one package gets updated we don't
assume all packages in the base are updated.
This commit mostly focuses on merging the install() and upgradePkgs()
functions to be more integrated. Instead of both making their own aur
queries they now both combine theyre needes into one query. This gives
us a speed up and allows us to easily print orphands, misising and out
of date in one clean block.
Currently sudoloop is run in parallel with the rest of the code this
causes the first sudo prompt to ask for a password while the code
continues running.
Instead hang on the first sudo, giving the chance for the user to enter
a password then continue the loop in the background.
Options such as --devel are now striped from the parser before handling
the command but the option is still processed so that config.devel would
be true.
Also changed `changedConfig` to a global in config.go
I have replaced the old install and dependancy algorithms with a new
design that attemps to be more pacaur like. Mostly in minimizing user
input. Ask every thing first then do everything with no need for more
user input.
It is not yet fully complete but is finished enough so that it works,
should not fail in most cases and provides a base for more contributors
to help address the existing problems.
The new install chain is as follows:
Source info about the provided targets
Fetch a list of all dependancies needed to install targets
I put alot of effort into fetching the dependancy tree
while making the least amount of aur requests as
possible. I'm actually very happy with how it turned out
and yay wil now resolve dependancies noticably faster
than pacaur when there are many aur dependancies.
Install repo targets by passing to pacman
Print dependancy tree and ask to confirm
Ask to clean build if directory already exists
Download all pkgbuilds
Ask to edit all pkgbuilds
Ask to continue with the install
Download the sources for each packagebuild
Build and install every package
using -s to get repo deps and -i to install
Ask to remove make dependancies
There are still a lot of things that need to be done for a fully working
system. Here are the problems I found with this system, either new or
existing:
Formating
I am not so good at formatting myself, I thought best to
leave it until last so I could get feedback on how it
should look and help implementing it.
Dependancy tree
The dependancy tree is usually correct although I have
noticed times where it doesnt detect all the
dependancies that it should. I have only noticed this
when there are circular dependancies so i think this
might be the cause. It's not a big deal currently
because makepkg -i installed repo deps for us which
handles the repo deps for us and will get the correct
ones. So yay might not list all the dependancies. but
they will get installed so I consider this a visual bug.
I have yet to see any circular dependancies in the AUR
so I can not say what will happend but I#m guessing that
it will break.
Versioned packages/dependencies
Targets and dependancies with version constriants such
as 'linux>=4.1' will not be checked on the aur side of
things but will be checked on the repo side.
Ignorepkg/Ignoregroup
Currently I do not handle this in any way but it
shouldn't be too hard to implement.
Conflict checking
This is not currently implemented either
Split Paclages
Split packages are not Handles properly. If we only
specify one package so install from a split package
makepkg -i ends up installing them all anyway. If we
specify more than one (n) package it will actually build the
package base n times and reinstall every split package
n times.
Makepkg
To get things working I decided to keep using the
makepkg -i method. I plan to eventually replace this
with a pacman -U based method. This should allow passing
args such as --dbpath and --config to aur packages
aswell as help solve some problems such as the split
packages.
Clean build
I plan to improve the clean build choice to be a little
more smart and instead of check if the directory exists,
check if the package is already build and if so skip the
build all together.
This allows us to use command line options such as '--root' and '-b' when
performing operations that use out alom handle.
Sadly this does not apply to passToMakepkg which will ignore options
such as '--root' and because we pass '-i' to makepkg it installs for us
using the default options.
Currently I have not planned a solution for this but one which I thought
of but not looked into is. Always call makepkg without arguments (except
from '--noconfirm' and others which might still be needed) and manage
the dependancies and post install outselves.
Another option might be to use makepkg's $PACMAN enviroment variable and
redirect the pacman calls to yay. Although I am unsure about both
stratergys they are just my current thoughts.
Also while editing the flow of cmd.go, I managed to refactor away all
os.Exit calls apart from the very last so it should be more clear as to
where the program exits.
Argument parsing now works mostly as expected for repo packages.
AUR packages are a little tricky becauce makepkg cant handle args such
as '--dbpath'.
Also out alpm handle does not read the commandline options so any
arguments relient on alpm will be ignored.
For now though it seems yay has gained back the functionality it once
had. While also having improved argument handling which should also be
expandable and make it easier to handle anything new that might have
been missed.
This reimplemens all operations yay previously supported:
'-S' 'Syu' 'Si' ect.
Currently the argument objects are not fully implemented with the code.
Theres alot of funky conversion from
argument object -> pkg, flags -> argument object
This is to just get back to the functionally we had before (almost).
I have not looked into it yet but alot of the time pacman flags get
passed to makepkg. this cases an error for most commands now because the
new system Passes all flags:
`yay -Syu` -> flags = '-S' '-y' '-u'
while the old system would have done:
`yay -Syu` -> op = '-Suy', flags = ''
So extra flags are no longer passed at all currently.
This means:
'yay -S aic94xx-firmware --noconfirm -b /tmp/pacutilesu2q6hw/tmp-pacman -d'
will no longer error and 'aic94xx-firmware' will be installed but the
database path change will not apply and the dep checking will not be
skipped.
passToPacman now takes and argParser as a paramater. And is implemented
for the simple cases in cmd.go. Although passToPacman is now left non
working in places which still try to usr the old call format and will
need to be reimplemented.
Try to minimise the useage of os.Exit
Apart from init, os.Exit is only used once as the final function call.
Now we can ensure there are no random exits hidding in the code. We can
also allow part of the code to error and continue on while also
remembering that we did error and return non 0 when we finally do reach
the os.Exit. This comes in very handy for trying to save the vcs info
after an error and ensuring that alpmHandle.Release is always called.
Some of the yay specific operations such as:
'-Qstats' '-Cd'
Break the standard pacamn has set.
Most yay specific specific options have been moved into their own
operator:
'-Y' '--yay'
'-Qstats' -> '-Y --stats'
'--Cd' -> '-Y --cleandeps'
'--gendb' -> '-Y --gendb'
Although:
'-G'
has been kepts and and alias has been added for is:
'--getpkgbuild'
This shouldnt feel too different for most people given that yay will
default to:
'-Y'
when no operator has been specified. So calls like:
'yay --gendb'
will still work as they did.
Internally handleConfig has been partially refactored. Now all that
remians in setters for config. All flags that did anything beond setting
config have been moved to handleYay(). This means less nasty
os.Exit()'s and nicer flow control.
The usage has also been updated. The options changes have been added but
the layout has also been tweaked a bit.
Lastly on the things not done:
Man page has not been updated
Shell completion has not been updated
This will be done in further commits as the usage is not guaranteed
to stay the same between now the completion of the argument system.
Currently the foundation for a new fuller argument parsing has been implemented in
parser.go. Most of the parsing is now done through the argParser object
instead of seperate arrays for options and packages. The rest of the
code still expects the old system so I have left most of the operations
unimplemented for now until I redo it with the new system. Currently
only '-S' and number menu have any functionality for testing purposes.
This new system parses arguments fully instead of just looking for
predefined strings such as:
'-Sqi' '-Siq'.
This allows:
'-Syu', '-S -y -u', '--sync -y -u'
to all be parsed as the same.
This system tries to be as similar to pacman as possible, eventually
aming to fully wrap pacman, allowing yay to be used instead of pacman in
all instances.
The current implementation is not as strict as pacman when checking
arguments. If you pass
--someinvalidflag to yay then yay will simply ignore it. The flag should
still be passed to pacman which should then cause an error.
Although operations '-S' '-R' '-U' ect. are checked to make sure you can not
try to use two operations at once.
conflicting flags such as:
'--quiet' and '--info'
will not raise an error and which options gains precedence is depend on
the implementation.
Another minor issue which is worth noting is. Due to the way double
arguments are parsed:
'-dd' '-cc' '--deps --deps'
if you pass the long version and the short version:
'-d --deps'
yay will not realize its a double argument. Meanwhile pacman will
reconise it when yay calls pacman.
Currently there are a few things that need to be done before this new
system can be fuly released:
Reimplement all operations to use to new parsing system so that
the new system is at least as functional as the old one
Strip yay specific flags before passing them to pacman
Move parts of config into the argument system and only use
config for options that are meant to be saved to disk
Move yay specific operations into its own operator '-Y'
Update documentation to show the altered syntax
