The easiest way that I have seen so far for doing so is to use tail and simply read everything except the first three bytes (start reading at the 4th byte), as follows:
tail --bytes=+4 text_file.txt > text_file-wo-bom.txtCategory: Linux
Perl One-Liner for Replacing Multiple Lines of a Text file With Multiple Lines of Text
When executing ‘search-and-replace’ commands on ASCII under Linux, Unix (or *nix) operating systems, sed works or most cases and makes for reasonably straightforward reading of the script.
If you want to replace multiple lines of text with multiple lines of text, following is a perl one-liner that does the trick and is much easier to wrangle than trying to do it in sed.
perl -i -pe "BEGIN{undef $/;} s:${EXISTING_LINES}:${REPLACEMENT_TEXT}:smg" file.txtAdding the Contents of a Source File to the Beginning of a Target File
Following is *nix a command that you can use to add the contents of a source text file to the start of another text file (the source file).
echo -e '0r <source_file_name>\nw' | ed -s <target_file_name>Creating a Samba Share and Configuring an Access Control List for a Shared Directory Under Linux
Often administrators would like to configure a Samba share that enables users to have the same access to any files within the share. Without some additional configuration, directories and files created by one user will not have the r/w permissions for other users that have access to that same share.
The end goal is to have a Samba share in which any new files are created with r/w permissions for a specific group to which all of the members of the share belong.
The first step is to determine whether the current running kernel supports ACL. To do this we check the configuration file that was used to build the kernel, looking for config keys that contain the string ‘POSIX_ACL’.
Determine the version of your current kernel
uname -a
Linux leviathan 2.6.32-358.2.1.el6.x86_64 #1 SMP Wed Mar 13 00:26:49 UTC 2013 x86_64 x86_64 x86_64 GNU/LinuxThen grep the config file for your kernel
grep POSIX_ACL /boot/config-2.6.32-358.2.1.el6.x86_64
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_FS_POSIX_ACL=y
CONFIG_XFS_POSIX_ACL=y
CONFIG_BTRFS_FS_POSIX_ACL=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_JFFS2_FS_POSIX_ACL=yIf the configuration for the filesystem type that you are using on the disk on which you want to configure the share indicates ‘y’, then you can continue to the next step. If not, you will need to recompile your kernel with ACL support.
Next, check the configuration entry in /etc/fstab for the filesystem that we want to use.
/dev/sda1 /usr/local2/ ext3 defaults 1 2In this case, our filesystem is not yet configured to use acl as it is not noted after the ‘defaults’ option.
Edit /etc/fstab and change that option to read ‘defaults,acl’ as follows:
/dev/sda1 /usr/local2/ ext3 defaults,acl 1 2Unmount, and then re-mount the filesystem.
Next, ensure that you already have a group configured for the users in question and that those users belong to that group. For this example our group will be ‘sharegroup’
Then, create the directory that you want to share via Samba and set the permissins and acl for it.
mkdir /usr/local2/shareChange the group ownership, permissions, and set the sgid flag.
chgrp sharegroup /usr/local2/share
chmod g+w share
chmod g+x share
chmod g+s shareNow we set the acl for the directory. The following sets the default group to ‘sharegroup’ with permissions of ‘rwx’.
setfacl -m d:g:sharegroup:rwx share/The last thing to do is to set up the share in Samba. Edit /etc/samba/smb.conf
[share]
comment = Shared Directory
path = /usr/local2/share
valid users = user1 user2
browseable = yes
writeable = yes
create mask = 2664
directory mask = 2775The key configurations are the create and directory mask which will ensure that files are created with the proper group permissions to enable all of the users the desired access.
Restart samba and you are finished.
If you copy an existing directory into the share directory you will need to modify the permissions so that it is available to all of the users who have access to that share.
Add sgid permissions to all of the sub directories:
find ./new_dir -type d -exec chmod g+s {} \;Add executable permissions for the group to all of the sub directories:
find ./new_dir -type d -exec chmod g+x {} \;Add write permissions to all files and directories in the new dir:
chmod -R g+w ./new_dirUse awk to Print from nth element to the End of the Line
If you want to extract from the nth token to the end of the line, following is how you can do that with awk:
Given a source file with the following:
line1 -- 01 0011 1
line2 -- 01 0011 2
line3 -- 01 0011 3
line4 -- 01 0011 4
line5 -- 01 0011 5
line6 -- 01 0011 6
line7 -- 01 0011 7
line8 -- 01 0011 8
line9 -- 01 0011 9
line10 -- 01 0011 10If you want remove the 1st, 2nd, and 3rd items from the list, you can use awk to set those fields to an empty value as follows
awk '{$1=$2=$3=""; print $0}' test.outWhich will result in:
0011 1
0011 2
0011 3
0011 4
0011 5
0011 6
0011 7
0011 8
0011 9
0011 10Firewall for Ubuntu 14.04 LTS
For whatever reason, Ubuntu 14.04 does not seem to come with a firewall.
There are however two packages which provide, both a firewall and a handy GUI front-end for it. UncomplicatedFireWall is the main package (ufw) and the GUI is gufw.
To install:
apt-get install gufwThis will install the front-end and the dependent packages
To turn it on:
ufw enableThe default is to block all incoming traffic.
To update and add your own firewall rules and allow incoming connections
gufwThe GUI is quite intuitive and allows advanced users the ability to create their own custom rules.
Generate a Random String of a Specified Size with a Shell Script
The following is a one-liner for generating a random string of a fixed size in bash, where the possible characters to use in the string are any digit, letter, and a newline.
By adding the newline, you are fairly sure to prevent getting one long line of text.
< /dev/urandom tr -dc "[:digit:][:alpha:][\n]" | head -c1000 > file.outOne-Liner for Converting CRLF to LF in Text Files
If you have text files created under DOS/Windows and need to convert the CRLF (carriage return and line feed) characters to LF (line feed) character, here is a quick one-liner.
cat file.txt | perl -ne 's/\x0D\x0A/\x0A/g; print' > file.txt.modYou can also use dos2unix, however, especially under Cygwin I have seen dos2unix fail without giving any meaningful information about why it was unable to complete the task. In that case, you can just do it by hand.
Parsing Command Line Arguments with getopt in Bash
When writing utility scripts in Bash it is tempting to simply pass positional arguments, use $1, $2, etc. and be done with it. However if you want to either share this utility with other members of your team and/or incorporate it into your system, it makes sense to implement your command line argument parsing in a more flexible and maintainable manner.
Using getopt you can very easily pass a variety of command line options and arguments.
Following is a link to a GitHub Gist with an example that illustrates the implementation of flags, options or arguments with values and long option names.
Vim Search and Replacing with Backreferences
It is often helpful to write search and replace commands that save segments of the matched text to use in the replacement string.
Source text:
This is some text (we want to change) with a phone number (301) 555-1234.
We want to remove the parenthesis, but only from the phone number string.In this example we have a text file that has a phone number in it and we want to remove the parenthesis that surround ONLY the area code in the phone number, plus the trailing space and replace it with the aread code plus a trailing “-“.
Following is the search and replace command. The forward slashes are separators in the search command. They will be omitted in the full explanation below:
:1,%s/(\([0-9]\+\))\s/\1-/gHere is it explained:
:1,%sTells, vi to execute the search and replace command from line 1 to the end of the document
(\([0-9]\+\))\sThe regex including the backreference that will find the area code including the parenthesis and a trailing space. The \( and \) parenthesis wrap the numerical part of the string that we are trying to save and mark it as a backreference. The \+ indicates that we one one or more of the numerical characters (the plus char must be escaped). The \s indicates a whitespace character the we also want to match.
\1-The characters that we want to use to replace what we have found. The \1 indicates we want to use the first backreference that was matched followed by a ‘-‘ character.
gTells, vi to execute the search and replace on all occurances found in a line.