Commit graph

144 commits

Author SHA1 Message Date
Marcel Moolenaar ba7bccb366 Suffix the cookie constants with ULL to silence warnings from compilers
that try to treat them as 32-bit values.
2014-10-01 20:37:15 +00:00
Marcel Moolenaar fb2f6c6c66 Update the usage message and the man page to account for the new long
options. Bump the version number to 20140927.
While here, use explicit fputc() calls to skip a line in the output.
This to avoid having to hunt for extra '\n' characters in the printf
format strings.

MFC after:	1 week
Relnotes:	yes
2014-09-27 17:58:03 +00:00
Marcel Moolenaar a53d83a210 Add 3 long options for getting information about mkimg itself:
--version	print the version of mkimg and also whether it's
		64- or 32-bit.
--formats	list the supported output formats separated by space.
--schemes	list the supported partitioning schemes separated by
		space.

Inspired by a patch from: gjb@

MFC after:	1 week
Relnotes:	yes
2014-09-27 04:53:51 +00:00
Marcel Moolenaar 32e570e701 Add baseline files for QCOW2. 2014-09-24 22:58:10 +00:00
Marcel Moolenaar e187064261 Finish QCOW version 2 and stop making it conditional.
We have a different ordering for the RC block(s) and L2 tables.
This is expected to be a non-issue, because everything is found
through file offsets in the corresponding RC table and L1 table.
Files that grow organically have RC blocks and L2 tables scattered
all over the place anyway.

The reason for the difference is that mkimg needs to be able to
write to a pipe. We can't seek forward and backward to fill in
the bits in non-sequential order.
2014-09-24 22:45:03 +00:00
Marcel Moolenaar 88ff6ba840 Update the baseline for QCOW version 1. A bug was found that rendered
the baseline invalid.
2014-09-24 15:15:14 +00:00
Marcel Moolenaar 0e651cfe2e Fix the creation of the L2 cluster table for version 1. The blkofs
variable was assigned the image offset in bytes and not in blocks
(i.e. sectors). This had image_data() return FALSE, which meant that
we didn't assign a cluster when we needed and also meant that we
didn't write parts of the L2 table when we should have. The result
being that the actual data clusters were written at the wrong offset.

Improve support for QCOW version 2. We're having the right layout
and even know how many refcnt blocks we need. All we need to do is
populate the refcnt blocks for every cluster we write and allocate
a cluster when we need a new refcnt block. The allocation part is
tricky in that it'll interleave with the assignment of clusters to
L2 tables and data. Since version 2 is not quite done, keep it
compiled out for now.
2014-09-24 15:14:01 +00:00
Marcel Moolenaar d60351df8f Clean the generated baseline files by adding them to CLEANFILES. 2014-09-24 04:17:29 +00:00
Marcel Moolenaar e286914f45 Rename the tests to something more meaningful. I spent way too long
trying to get the test name right, failed, gave up and used a sequence
number instead. When I realized it wasn't because of the number of
underscores in the name that I really started to think. I didn't have
braces around the variable names ...

Thus: test_1 is now called apm_1x1_512_qcow, which gives you all you
need to run mkimg by hand.

Dumb-ass: marcel
2014-09-22 16:52:09 +00:00
Marcel Moolenaar 66829acbb3 Don't update the baseline file when the result of the test is identical
to the baseline. Since we don't run gzip with the -n option, the output
of gzip varies for identical result files if and when they are created
at different time. Ouch...

Rather than add -n and commit a 600K+ diff for the changes to all the .uu
files, it's less of a churn to uudecode and gunzip the baseline file and
compare that to the new result file to determine if the baseline file
needs to be updated.

This way, "atf-sh mkimg.sh rebase" can be run as many times as people like
and a subsequent "svn status" will not show unnecessary diffs.
2014-09-22 16:37:37 +00:00
Marcel Moolenaar c9b97e3bda Don't echo '# $FreeBSD$' as the first line into the .uu file. Keyword
substitution applies to this file, including the echo command. Avoid
the match (and substitution) by breaking the string up into 3 parts.
2014-09-22 15:27:23 +00:00
Marcel Moolenaar 572950ab03 Update the unit tests to include the QCOW (version 1) format.
This is a good safety net for when V2 is added.
2014-09-22 15:08:58 +00:00
Marcel Moolenaar 38501a4664 Add support for QCOW version 1. Version 2 is partially implemented.
And because of that, it's entirely disabled for now. Both versions
are similar enough that a single header definition works for both
of them. The only "diverting" side-effect is that the union of the
two is larger than the official V1 header.

What this means for our V1 support is that we can't put the L1 table
adjacent to the V1 header (i.e. at offset 0x30 in the file), unless
we revert to hackery and klugery. Let's not. Instead, we align the L1
table at the cluster boundary. This is in line with the V2 layout and
perfectly ok for V1 anyway (ok -- as far as I've seen so far).
Due to the alignment, our V1 image seems to be 1 cluster larger than
the V1 image created by qemu-img (on average).

Compression of the clusters is not supported at this time.

MFC after:	2 months
2014-09-22 15:05:28 +00:00
Marcel Moolenaar cd0b4a3c40 Add unit tests for mkimg(1): 2014-09-20 21:02:54 +00:00
Marcel Moolenaar 53fcdb2d1a Fix partition alignment and image rounding when any of -P (block size),
-T (track size) or -H (number of heads) is given:
o   scheme_metadata() always rounded to the block size.  This is not
    always valid (e.g. vtoc8 that must have partitions start at cylinder
    boundaries).
o   The bsd and vtoc8 schemes "resized" the image to make it match the
    geometry, but since the geometry is an approximation and the size
    of the image computed from cylinders * heads * sectors is always
    smaller than the original image size, the partition information ran
    out of bounds.

The fix is to have scheme_metadata() simply pass it's arguments to the
per-scheme metadata callback, so that schemes not only know where the
metadata is to go, but also what the current block address is. It's now
up to the per-scheme callback to reserve room for metadata and to make
sure alignment and rounding is applied.

The BSD scheme now has the most elaborate alignment and rounding. Just
to make the point: partitions are aligned on block boundaries, but the
image is rounded to the next cyclinder boundary.

vtoc8 now properly has all partitions aligned (and rounded) to the
cyclinder boundary.

Obtained from:	Juniper Networks, Inc.
MFC after:	3 days
2014-09-19 23:16:02 +00:00
Marcel Moolenaar 752bf46955 Add support for adding empty partition entries. I.e. skip partition
numbers or names. This gives more control over the actual layout and
helps to construct BSD disklabels with /usr or /var at dedicated
partitions.

Obtained from:	Juniper Networks, Inc.
MFC after:	3 days
Relnotes:	yes
2014-09-12 20:05:08 +00:00
Marcel Moolenaar de3307644e Be compatible with boot code that starts right after the disk label in
the second sector by only clearing the amount of bytes needed for the
disklabel in the second sector. Previously we were clearing exactly 1
sector worth of bytes and as such writing over boot code that may have
been there.
Since we do support more than 8 partitions, make sure to set all fields
in d_partitions. For the first 8 partitions this is unneeded, but for
partitioons 9 and up this compensates for the fact that we don't clear
an entire sector anymore.
Obviously, one cannot use more than 8 partitions when using boot code
that starts right after the disk label.

Relevant GRNs:
107879 - Employ unused bytes after the disklabel in the second sector.
189500 - Revert the part of change 107879 that employs the unused bytes
	 after the disklabel in the 2nd sector for boot code.

Obtained from:	Juniper Networks, Inc.
MFC after:	3 days
2014-09-12 04:15:35 +00:00
Marcel Moolenaar 04ca14cc2a Fix checksum calculation:
1.  Iterate over all partitions counted in the label, which can be more
    than the number of partitions given to mkimg(1).
2.  Start the checksum from the beginning of the label; not the beginning
    of the bootarea.

Tested with bsdlabel(8).

MFC after:	3 days
2014-09-12 03:54:16 +00:00
Marcel Moolenaar 6219349311 Create a redundant grain directory and table. Previously we were
cheating by assigning the same sector offset to both directories,
but it seems that VirtualBox doesn't like that. Neither does
qemu from the looks of it.  We now actually write the directory
and table twice.

MFC after:	3 days
2014-08-09 04:47:12 +00:00
Marcel Moolenaar 7ea7f92899 Fix builds on older FreeBSD versions and/or non-FreeBSD machines:
don't use _Static_assert unconditionally.
2014-07-23 18:05:39 +00:00
Marcel Moolenaar a1afbf00f6 Add support for the fixed image type. The fixed image is effectively
a raw image with a VHD footer appended. There's little value that I
can see to use the fixed image type, but in order to make VHD images
for use by Microsoft's Azure platform, they must be fixed VHD images.

Support has been added by refactoring the code to re-use common code
and by adding a second output format structure.  To created fixed VHD
images, specify "vhdf" as the output format.
2014-07-17 16:33:38 +00:00
Xin LI a11d210986 Add a bandaid to fix GCC build (on sparc64 et al). 2014-07-15 23:35:06 +00:00
Marcel Moolenaar a0f136e1c5 Add image_data() for checking whether a sequence of blocks has data.
Use this for VHD and VMDK to avoid allocating space in the image
for empty sectors.

Note that this negatively affects performance because mkimg uses a
temporary file for the intermediate storage. When mkimg has better
internal book keeping, performance can be significantly improved.
2014-07-15 04:39:23 +00:00
Marcel Moolenaar 5d4393ed1f Make this compile on older FreeBSD versions that don't have
APM_ENT_TYPE_APPLE_BOOT.
2014-07-11 01:49:25 +00:00
Marcel Moolenaar 264f86e1ad Document the -y option as a unit test option.
Add missing -v (and -y) to the usage message.

Requested by: eadler@
2014-07-04 18:47:25 +00:00
Marcel Moolenaar f3582a728d Add VHD support to mkimg(1). VHD is used by Xen and Microsoft's Hyper-V
among others.

Add an undocumented option for unit testing (-y). When given, the image
will have UUIDs and timestamps synthesized in a way that gives identical
results across runs. As such, UUIDs stop being unique, globally or
otherwise.

VHD support requested by: gjb@
2014-07-03 20:31:43 +00:00
Marcel Moolenaar 91e9fb22ea Add a ful stop after FreeBSD's version in the history section.
Pointed out by: brueffer@ (thanks!)
2014-07-02 14:34:01 +00:00
Marcel Moolenaar 73f28b23c4 Prepare for merging to stable/10: update the history section. 2014-07-02 04:06:06 +00:00
Joel Dahl b6829dc82a Minor mdoc improvements. 2014-06-06 19:00:43 +00:00
Marcel Moolenaar aa30ba04c3 Create our temporary file in $TMPDIR, if the environment variable
is set. /tmp otherwise.

Submitted by:   Dan McGregor <danismostlikely@gmail.com>
2014-05-22 20:24:30 +00:00
Marcel Moolenaar 762ff43901 Fix CID 1204379 (vtoc8.c) & CID 1204380 (bsd.c): Cast ncyls to lba_t
before multiplying the 32-bit integrals to avoid any possibility of
truncation before widening. Not a likely scenario to begin with...
2014-05-21 17:39:49 +00:00
Marcel Moolenaar adc991ea42 Fix CID 1204394: Use strncpy(3) instead of strcpy(3). Note that it's
ok to not have the name and type strings terminated.
2014-05-21 17:38:56 +00:00
Marcel Moolenaar 645c72194e Fix CID 1215124: Handle errors properly. 2014-05-21 17:38:14 +00:00
Marcel Moolenaar a513818762 Fix CID 1215125: fstat(2) returns -1 on error and sets errno. It does
not return the error (oops).
2014-05-21 17:37:22 +00:00
Marcel Moolenaar 9746454f54 Fix CID 1215128: Free the allocated buf when image_set_size()
returns and error and we return from bsd_write().
2014-05-21 17:36:12 +00:00
Marcel Moolenaar bce9a24a0e Fix CID 1215129: move the call to lseek(2) before the call to malloc(3)
so that the error path (taken due to lseek(2) failing) isn't leaking
memory.
2014-05-21 17:34:50 +00:00
Marcel Moolenaar f0e9dced5c MFuser/marcel/mkimg:
Add support for different output formats:
1.  The output file that was previously written is now called the raw format.
2.  Add the vmdk output format to create VMDK images.

When the format is not given, the raw output format is assumed.
2014-05-15 19:19:57 +00:00
Marcel Moolenaar 789a10b106 Add mkimg_write() which combines lseek(2) and write(2) and uses
sector granularity for both offset and length. Have all schemes
use mkimg_write() instead of mkimg_seek() followed by write(2).

Now that schemes don't use lseek(2) nor write(2) directly, it's
easier to support output formats other than raw disks.
2014-05-06 21:54:05 +00:00
Marcel Moolenaar c562fda42a In apm_write(), both fd and imgsz are referenced, so don't mark the
arguments as unused.
2014-05-06 20:34:21 +00:00
Nathan Whitehorn 2cbc36ab59 Add freebsd-boot to recognized partition types. 2014-05-01 03:24:20 +00:00
Marcel Moolenaar 9e4108268e Fix build on FreeBSD 7 where:
1.  DOSPTYP_FAT32 is not defined in <sys/diskmbr.h>
2.  uuid_enc_le() does not exist in libc.
2014-03-29 23:46:01 +00:00
Marcel Moolenaar f529e2e0a5 Fix build on FreeBSD 8 where partition types for nandfs do not exist. 2014-03-29 22:10:54 +00:00
Marcel Moolenaar 9bc923cbde Fix build on FreeBSD 9 where <sys/pc98.h> has the same defines as
<sys/diskmbr.h> and not the unique defines introduced later.
2014-03-29 22:02:25 +00:00
Marcel Moolenaar a5eb4ea3ee Add mkimg, a utility for making disk images from raw partition contents.
The partitioning scheme can be one of the schemes supported by gpart.

Reviewed by:	sjg
Obtained from:	Juniper Networks, Inc.
2014-03-29 19:03:10 +00:00