List Recursively Files in Big Directory With Continuation Token
Chapter 3 Working with Files and Directories
The SunOS command line is used to manipulate files and directories. You type in the file and directory names in conjunction with SunOS commands to carry out specific operations. This is different than using the OpenWindows File Manager, where files are displayed as icons that can be clicked on and moved, and commands are selected from menus.
This chapter introduces you to the concepts and procedures used to work with files and directories from the SunOS command line. These operations apply to any SunOS command line, whether you are using a Shell or Command Tool in the OpenWindows environment or are logged in from a remote terminal. To fully make use of the SunOS operating system it is essential for you to understand the concepts presented in this chapter.
3.1 File Concepts
The file is the basic unit in the SunOS operating system. Almost everything is treated as a file, including:
-
Documents--These include text files, such as letters or reports, computer source code, or anything else that you write and want to save.
-
Commands--Most commands are executable files; that is, they are files you can execute to run a particular program. For example, the date command that you saw in the previous chapter, which executes a program that provides the current date, is an executable file.
-
Devices--Your terminal, printer, and disk drive(s) are all treated as files.
-
Directories--A directory is simply a file that contains other files.
The following section explains the commands available for creating, listing, copying, moving, and deleting files. You'll also see how to list the contents of a file and how to determine the nature of a file.
3.2 Using File Commands
Each of the commands presented in this section includes an example of how the command is used. Try the examples as you read the text. This practice will make the commands and their respective concepts easier to understand and remember.
3.2.1 Before You Begin
Before you start experimenting with files, make sure that you are in your home directory. This is a directory established for you by your system administrator when your account was created. If you perform the tasks shown in the following examples from your home directory, you'll be less likely to create, copy, move, or (worst of all) delete files within portions of the system that other users expect to remain unchanged.
To make certain that you are indeed in your home directory, type the cd (change directory) command by itself. This moves you to your home (default) directory. Then type the pwd (print working directory) command to display your current location within the filesystem. The directory displayed is your home directory:
$ cd $ pwd /export/home/username
In this example, the user's home directory is /export/home/ username, where username is the name of the user owning the home directory.
3.2.2 Creating a Test File
Use the touch command to create an empty file. If a file by the name you specify doesn't already exist, the touch command creates an empty file (if the file already exists, touch updates the last file access time).
3.2.3 Listing Files (ls)
Now list the file with the ls command to verify that you've created it:
When you enter the ls command by itself, it lists all the files in your current location. If you enter the ls command with a specific file name, it lists only that file, if the file exists.
For more information on the ls(1) command, refer to the man Pages(1): User Commands.
3.2.4 Copying Files (cp)
Use the cp command to copy tempfile to a file called copyfile:
Now try listing both files. Notice that both names end with the characters "file." You can use the wildcard character, asterisk (*), to stand for any character or sequence of characters. Therefore, the command ls *file should list both tempfile and copyfile (and any other file in this directory with a name that ends with file):
$ ls *file copyfile tempfile
Notice that copyfile is listed first. Files are listed in alphabetical order. (Capital letters and numbers precede lowercase letters.)
For detailed information on the cp(1) command, refer to the man Pages(1): User Commands.
3.2.5 Moving and Renaming Files (mv)
You can both move and rename files using the same command, mv (move). In this example, use the mv command to rename tempfile to emptyfile:
$ mv tempfile emptyfile $
Now list both files again to verify the change:
$ ls *file copyfile emptyfile
As you can see, tempfile is replaced by emptyfile.
For more information on the mv(1) command, refer to the man Pages(1): User Commands.
3.2.6 Deleting Files (rm)
Finally, use the rm (remove) command to delete copyfile, and verify the result with the ls command:
$ rm copyfile $ ls *file emptyfile
Caution -
Once you delete a file, it is gone for good. Unless there is a backup copy, the file cannot be restored. Be careful when using the rm command, and be particularly careful when using rm with the wildcard character (*). Files removed with rm cannot be recovered.
For more detailed information on the rm(1) command, refer to the man Pages(1): User Commands.
3.2.7 Displaying File Contents (more, cat)
Use the more command to display the contents of a file. Type more followed by the name of the file to be displayed. The contents of the file scrolls down the screen. If the file is longer than one screen, this message appears:
--More--(nn%) [Press space to continue, `q' to quit.]
where nn is the percentage of the file already displayed.
You can also use the cat command to display the contents of a file, but it flashes through the entire file rapidly without pausing. The cat (concatenate) command is more often used to join two or more files into one large file, as in this example:
$ cat file1 file2 file3 > bigfile $ ls *file bigfile file1 file2 file3 $
For further information on the more(1) or cat(1) commands, refer to the man Pages(1): User Commands.
3.2.8 Displaying File Type (file)
Some files, such as binary files and executable files, are not printable and cannot be displayed on the screen. The file command can be handy if you're not sure of the file type.
Use the file command to show the file type:
$ file copyfile copyfile: ascii text
3.3 Directories and Hierarchy
By now you know how to list, copy, rename, and delete files. However, you may be wondering about larger issues. Where are these files located? This section discusses the directory hierarchy. Read the following narrative carefully, and then try the examples in the sections that follow.
3.3.1 Directory Hierarchy
Files are grouped into directories, which are themselves organized in a hierarchy. At the top of the hierarchy is the "root" directory, symbolized by "/".
As shown in the following example, Figure 3-1, each directory in the file system can have many directories within it. The convention is to distinguish directory levels with the / character. With this in mind, notice that the directory / (root) contains the subdirectories /usr, /bin, /home and /lib, among others. The subdirectory /home contains user1, user2, and user3.
You specify directories (and files within them) by including the names of the directories they're in. This is called a path name. For example, the path name for the user3 directory in the illustration above is /home/user3.
Figure 3-1 File System Hierarchy
All subdirectory and file names within a directory must be unique. However, names within different directories can be the same. For example, the directory /usr contains the subdirectory /usr/lib. There is no conflict between /usr/lib and /lib because the path names are different.
Path names for files work exactly like path names for directories. The path name of a file describes that file's place within the file system hierarchy. For example, if the /home/user2 directory contains a file called report5, the path name for this file is /home/user2/report5. This shows that the file report5 is within the directory user2, which is within the directory home, which is within the root (/) directory.
Directories can contain only subdirectories, only files, or both.
3.3.2 Print Working Directory (pwd)
The command pwd (print working directory) tells you where you are in the file system hierarchy:
Your output will look somewhat different from that in the example, as your directory structure will be different. Remember that your working directory is your current location within the file system hierarchy.
3.3.3 Your Home Directory
Every user has a home directory. When you first open the Command Tool or Shell Tool window in the OpenWindows environment, your initial location (working directory) is your home directory. This directory is established for you by the system administrator when your account is created.
3.3.4 Change Working Directory (cd)
The cd (change directory) command allows you to move around within the file system hierarchy:
$ cd /usr/lib $ pwd /usr/lib
When you type the cd command by itself, you return to your home directory. For example, if your home directory was /home/user1:
In the C shell, the tilde (~) is used as a shortcut for specifying your home directory. For example, you would type the following to change to the subdirectory music within your home directory:
You can also use this shortcut to specify another user's home directory. For example:
where username is another user's login name, would change to that user's home directory.
Note -
If you are using the Bourne shell, the ~ shortcut will not work.
If you are using the Bourne shell, it may be possible that your system administrator has configured the system so that you can type $home to specify your home directory. If this is the case, then typing:
changes you to the subdirectory music in your home directory. Likewise, typing:
changes you to the specified user's home directory, where username represents another user's login name.
The directory immediately "above" a subdirectory is called the parent directory. In the preceding example, /home is the parent directory of /home/user1. The symbol .. ("dot-dot") represents the parent directory. Therefore, the command cd .. changes the working directory to the parent directory, as in this example:
$ pwd /home/user1 $ cd .. $ pwd /home
Suppose your current working directory is /home/user1 and you want to work with some files in /home/user2. Here is a useful shortcut:
$ pwd /home/user1 $ cd ../user2 $ pwd /home/user2
../user2 tells the system to look in the parent directory for user2. As you can see, this is much easier than typing the entire path name /home/user2.
3.3.5 Creating a Directory (mkdir)
It is easy to create a new directory. Type the mkdir command followed by the name of the new directory:
$ mkdir veggies $ cd veggies $ mkdir broccoli $ cd broccoli $ pwd /home/user2/veggies/broccoli
3.3.6 Relative Path Names
The full path name of a directory or a file begins with a slash (/) and describes the entire directory structure between that file (or directory) and the root directory. However, you can often use a much shorter name which defines the file or directory relative to the current working directory.
When you are in a parent directory, you can move to a subdirectory using only the directory name and not the full path name. In the previous example, the command cd veggies uses the relative path name of the directory veggies. If the current working directory is /home/user2, the full path name of this directory is /home/user2/veggies.
Try creating several different subdirectories, and then move around within this directory structure. Use both full path names and relative path names, and confirm your location with the pwd command.
3.3.7 Moving and Renaming Directories
You rename a directory by moving it to a different name. Use the mv command to rename directories:
$ pwd /home/user2/veggies $ ls broccoli $ mv broccoli carrots $ ls carrots
You can also use mv to move a directory to a location within another directory:
$ pwd /home/user2/veggies $ ls carrots $ mv carrots ../veggies2 $ ls ../veggies2 carrots
In this example, the directory carrots is moved from veggies to veggies2 with the mv command.
3.3.8 Copying Directories
Use the cp -r command to copy directories and the files they contain:
$ cp -r veggies veggies3 $
This command copies all files and subdirectories within the directory veggies to a new directory veggies3. This is a recursive copy, as designated by the -r option. If you attempt to copy a directory without using this option, you will see an error message.
3.3.9 Removing Directories (rmdir)
To remove an empty directory, use the rmdir command as follows:
If the directory still contains files or subdirectories, the rmdir command will not remove the directory.
Use rm -r (adding the recursive option -r to the rm command) to remove a directory and all its contents, including any subdirectories and their files, as follows:
Caution -
Directories removed with the rmdir command cannot be recovered, nor can directories and their contents removed with the rm -r command.
3.4 Looking at Differences Between Files (diff)
It often happens that different people with access to a file make copies of the file and then edit their copies. diff will show you the specific differences between versions of an ASCII file. The command:
$ diff leftfile rightfile
scans each line in leftfile and rightfile looking for differences. When it finds a line (or lines) that differ, it determines whether the difference is the result of an addition, a deletion, or a change to the line, and how many lines are affected. It tells you the respective line number(s) in each file, followed by the relevant text from each.
If the difference is the result of an addition, diff displays a line of the form:
where l is a line number in leftfile and r is a line number in rightfile.
If the difference is the result of a deletion, diff uses a d in place of a; if it is the result of a change on the line, diff uses a c.
The relevant lines from both files immediately follow the line number information. Text from leftfile is preceded by a left angle bracket (<). Text from rightfile is preceded by a right angle bracket (>).
This example shows two sample files, followed by their diff output:
$ cat sched.7.15 Week of 7/15 Day: Time: Action Item: Details: T 10:00 Hardware mtg. every other week W 1:30 Software mtg. T 3:00 Docs. mtg. F 1:00 Interview $ cat sched.7.22 Week of 7/22 Day: Time: Action Item: Details: M 8:30 Staff mtg. all day T 10:00 Hardware mtg. every other week W 1:30 Software mtg. T 3:00 Docs. mtg. $ diff sched.7.15 sched.7.22 1c1 < Week of 7/15 --- > Week of 7/22 4a5 > M 8:30 Staff mtg. all day 8d8 < F 1:00 Interview
If the two files to be compared are identical, there is no output from diff.
The diff(1)command has many more options than those discussed here. For more information, refer to the man Pages(1): User Commands.
3.4.1 Comparing Three Different Files (diff3)
If you have three versions of a file that you want to compare at once, use the diff3 command as follows:
$ diff3 file1 file2 file3
diff3 compares three versions of a file and publishes disagreeing ranges of text flagged with these codes:
==== all three files differ
====1 file1 is different
====2 file2 is different
====3 file3 is different
3.4.2 Using bdiff on Large Files
If you are comparing very large files, use bdiff instead of diff. Both programs work in a similar manner:
$ bdiff leftfile rightfile
Use bdiff instead of diff for files longer than 3500 lines or so.
3.5 Looking Up Files (find)
The find command searches for files that meet conditions you specify, starting from a directory you name. For example, you might search for filenames that match a certain pattern or that have been modified within a specified time frame.
Unlike most commands, find options are several characters long, and the name of the starting directory must precede them on the command line as follows:
where directory is the name of the starting directory and options represents the options for the find command.
Each option describes a criterion for selecting a file. A file must meet all criteria to be selected. Thus, the more options you apply, the narrower the field becomes. The -print option indicates that you want the results to be displayed. (As described later on, you can use find to run commands. You may want find to omit the display of selected files in that case.)
The -name filename option tells find to select files that match filename. Here filename is taken to be the rightmost component of a file's full path name. For example, the rightmost component of the file /usr/lib/calendar is calendar. This portion of a file's name is often called its base name.
For example, to see which files within the current directory and its subdirectories end in s, type:
$ find . -name '*s' -print ./programs ./programs/graphics ./programs/graphics/gks ./src/gks $
Other options include:
-name filename
Selects files whose rightmost component matches filename. Surround filename with single quotes if it includes filename substitution patterns.
-user userid
Selects files owned by userid. userid can be either a login name or user ID number.
-group group
Selects files belonging to group.
-m -time n
Selects files that have been modified within n days.
-newer checkfile
Selects files modified more recently than checkfile.
You can specify an order of precedence by combining options within (escaped) parentheses (for example, \(options\) ). Within escaped parentheses, you can use the -o flag between options to indicate that find should select files that qualify under either category, rather than just those files that qualify under both categories:
$ find . \( -name AAA -o -name BBB \) -print ./AAA ./BBB
You can invert the sense of an option by prepending an escaped exclamation point. find then selects files for which the option does not apply:
$ find . \!-name BBB -print ./AAA
You can also use find to apply commands to the files it selects with the
-exec command '{}' \;
option. This option is terminated with an escaped semicolon (\;). The quoted braces are replaced with the filenames that find selects.
As an example, you can use find to automatically remove temporary work files. If you name your temporary files consistently, you can use find to seek them out and destroy them wherever they lurk. For example, if you name your temporary files junk or dummy, this command will find them and remove them:
$ find . \( -name junk -o -name dummy \) -exec rm '{}' \; For more information on find(1), refer to the man Pages(1): User Commands.
3.6 File and Directory Security
Note -
Read this section carefully. A clear understanding of file permissions is often important in day-to-day work.
File permissions help to protect files and directories from unauthorized reading and writing. Often you will have files you wish to allow others to read but not change. In other cases, you may have executable files (programs) to share. File permissions allow you to control access to your files.
These are the basic file and directory permission types:
-
r - read permission. A file must be readable to be examined or copied. A directory must be readable for you to list its contents.
-
w - write permission. A file must be writable in order for you to modify it, remove it, or rename it. A directory must be writable in order for you to add or delete files in it.
-
x - execute permission. A file with executable permissions is one you can run, such as a program. A directory must be executable for you to gain access to any of its subdirectories.
There are three categories of users for which you can set permissions:
-
Self - The user
-
Group - Other users within the same group as the user (for example, all accounting users). Groups are established and maintained by the system administrator.
-
Others - Everyone else
3.6.1 Displaying Permissions and Status (su ls -l )
You have already used the ls command to list files. The ls command has many options. Use the -l option to display a long format list. Files and directories are listed in alphabetical order. Figure 3-2 illustrates this method for displaying files:
Figure 3-2 Displaying Permissions and Status
The very first character on the line indicates the file type. A dash (-) is an ordinary file; a d indicates a directory, and other characters can indicate other special file types.
The next nine characters indicate the permissions for the file or directory. The nine characters consist of three groups of three, showing the permissions for the owner, the owner's group, and the world, respectively. The permissions for emptyfile are rw-r--r--, indicating that the owner can read and write this file, everyone can read it, and no one can execute it. The permissions for the directory veggies2 are rwxr-xr-x, indicating that everyone has read and execute permissions, but only the owner can write to it.
In addition to file permissions, the display shows the following information:
-
Number of links to this file or directory
-
Name of the owner (user2 in this case)
-
Number of bytes (characters) in the file
-
Date and time the file or directory was last updated
-
Name of the file or directory
Use the cd command to move to your home directory, and try the ls -l command. Your results will differ from the example, of course.
Now try typing a command such as the following:
where dirname is the name of an actual directory within your file system. When you give the name of a directory, the ls -l command prints information on all the files and directories (if any) within that directory.
3.6.2 Listing "Hidden" Files (ls -a )
There are some files that are not listed by the ordinary ls command. These files have names beginning with the character . (called "dot"), such as .cshrc, .login and .profile. Use the ls -a command to list these dot files:
$ ls -a . .. .cshrc .login .profile emptyfile
Notice that the files beginning with . are listed before the other files. There are two special files in this listing: the file .is the reference for the current directory, and the file .. is the reference for the parent directory.
Generally speaking, files that begin with . are used by system utilities and are not usually modified by the user. There are a few exceptions to this.
3.6.3 Changing Permissions (chmod)
Use the chmod command to change permissions for a file or directory. You must be the owner of a file or directory, or have root access, to change its permissions. The general form of the chmod command is:
where permissions indicates the permissions to be changed and name is the name of the affected file or directory.
The permissions can be specified in several ways. Here is one of the forms which is easiest to use:
-
Use one or more letters indicating the users involved:
-
u (for the user)
-
g (for group)
-
o (for others)
-
a (for all three of the above categories)
-
-
Indicate whether the permissions are to be added (+) or removed (-).
-
Use one or more letters indicating the permissions involved:
-
r (for read)
-
w (for write)
-
x (for execute)
-
In the following example, write permission is added to the directory carrots for users belonging to the same group (thus, permissions is g+w and name is carrots):
$ ls -l carrots drwxr-xr-x 3 user2 1024 Feb 10 11:15 carrots $ chmod g+w carrots $ ls -l carrots drwxrwxr-x 3 user2 1024 Feb 10 11:15 carrots $
As you can see, the hyphen (-) in the set of characters for group is changed to a w as a result of this command.
To make this same directory unreadable and unexecutable by other users outside your group (permissions is o-rx), you would enter the following:
$ ls -l carrots drwxrwxr-x 3 user2 1024 Feb 10 11:15 carrots $ chmod o-rx carrots $ ls -l carrots drwxrwx--- 3 user2 1024 Feb 10 11:15 carrots $
Now, the r (for read) and the x (for execute) in the set of characters for other users are both changed to hyphens (-).
When you create a new file or directory, the system automatically assigns permissions.
In general, the default settings for new files are:
-rw-r--r--
and for new directories are:
drwxr-xr-x
So, to make a new file turnip executable by its owner (user2), you would enter the following:
$ ls -l turnip -rw-r--r-- 3 user2 1024 Feb 10 12:27 turnip $ chmod u+x turnip $ ls -l turnip -rwxr--r-- 3 user2 1024 Feb 10 12:27 turnip $
If you want to affect all three categories of users at once, use the -a option. To make a new file garlic executable by everyone, you would enter the following:
$ ls -l garlic -rw-r--r-- 3 user2 1024 Feb 10 11:31 garlic $ chmod a+x garlic $ ls -l garlic -rwxr-xr-x 3 user2 1024 Feb 10 11:31 garlic $
As a result, the x indicator appears in all three categories.
You can also change permissions for groups of files and directories using the * wildcard character. For example, you would enter the following to change the permissions for all the files in the current directory veggies so that the files can be written by you alone:
$ pwd /home/user2/veggies $ ls -l -rwxrwxrwx 3 user2 21032 Feb 12 10:31 beats -rwxrwxrwx 2 user2 68 Feb 10 11:09 corn -rwxrwxrwx 3 user2 12675 Feb 08 09:31 garlic -rwxrwxrwx 1 user2 1024 Feb 14 16:38 onions $ chmod go-w * $ ls -l -rwxr-xr-x 3 user2 21032 Feb 12 10:31 beats -rwxr-xr-x 2 user2 68 Feb 10 11:09 corn -rwxr-xr-x 3 user2 12675 Feb 08 09:31 garlic -rwxr-xr-x 1 user2 1024 Feb 14 16:38 onions $
The pwd command is included in this example to illustrate that the directory on which you perform this chmod operation must be the current directory.
3.6.4 Setting Absolute Permissions
Up to this point, the discussion on permissions has only included using the chmod command to change permissions relative to their current settings. Using a different form of the chmod command, which applies numeric codes to specify permissions, you can set the permissions for a file or directory absolutely.
The syntax for this usage of the chmod command is:
chmod numcode name
where numcode is the numeric code and name is the name of the file or directory for which you are changing permissions.
The complete numeric code consists of three numbers. One number is used for each of the three categories: user, group, and others. For example the following command sets absolute read, write, and execute permissions for the user and the group, and execute permissions only for others:
Table 3-1 illustrates how the permissions described for garlic are represented by the code 771.
Table 3-1 Permissions for garlic
| Permission | User | Group | Others |
|---|---|---|---|
| Read | 4 | 4 | 0 |
| Write | 2 | 2 | 0 |
| Execute | 1 | 1 | 1 |
| Total | 7 | 7 | 1 |
Each of the columns in Table 3-1 represents one of the categories: user, group, and others. To set read permissions, you add 4 to the appropriate column. To set write permissions, you add 2. To add execute permissions, you add 1. The total in all three columns in the last row of the table is the complete numeric code.
The following is another example of this method for setting absolute permissions, with the ls -l command included to demonstrate the results:
$ ls -l onion -rw-r--r-- 3 user2 1024 Feb 10 11:46 onion $ chmod 755 onion $ ls -l onion -rwxr-xr-x 3 user2 1024 Feb 10 11:48 onion $
The permissions for the file onion are set so that the user can read, write, and execute; group members can read and execute; and others can also read and execute. Table 3-2 provides the breakdown of the numeric code used to set the permissions for onion.
Table 3-2 Permissions for onion
| Permission | User | Group | Others |
|---|---|---|---|
| Read | 4 | 4 | 4 |
| Write | 2 | 0 | 0 |
| Execute | 1 | 1 | 1 |
| Total | 7 | 5 | 5 |
Of course, to provide read, write, and execute permissions for the file cabbage to yourself, your group, and all other users, you would enter the following:
$ ls -l cabbage -rw-r--r-- 3 user2 1024 Feb 10 11:51 cabbage $ chmod 777 cabbage $ ls -l cabbage -rwxrwxrwx 3 user2 1024 Feb 10 11:53 cabbage $
Table 3-3 provides the breakdown for this example.
Table 3-3 Permissions for cabbage
| Permission | User | Group | Others |
|---|---|---|---|
| Read | 4 | 4 | 4 |
| Write | 2 | 2 | 2 |
| Execute | 1 | 1 | 1 |
| Total | 7 | 7 | 7 |
The numeric code 777 represents the maximum level of permissions you can provide.
Similar to changing relative permissions, you can also use the wildcard character * to set absolute permissions for all in the files in the current directory. For example, to set absolute permissions for all files in the current directory veggies so that you have read, write, and execute permissions; your group has read and execute permissions; and all other users have execute permissions only, you would enter the following:
$ pwd /home/user2/veggies $ ls -l -rwxrwxrwx 3 user2 21032 Feb 12 10:31 beats -rwxrwxrwx 2 user2 68 Feb 10 11:09 corn -rwxrwxrwx 3 user2 12675 Feb 08 09:31 garlic -rwxrwxrwx 1 user2 1024 Feb 14 16:38 onions $ chmod 751 * $ ls -l -rwxr-x--x 3 user2 21032 Feb 12 10:31 beats -rwxr-x--x 2 user2 68 Feb 10 11:09 corn -rwxr-x--x 3 user2 12675 Feb 08 09:31 garlic -rwxr-x--x 1 user2 1024 Feb 14 16:38 onions $
The pwd command is included in this example to illustrate that the directory on which you perform this operation must be the current directory. The ls -l command is shown only to illustrate the changes in permissions. When setting absolute permissions, it's not necessary to know what the permissions are currently.
For more information on the chmod(1) command, refer to the man Pages(1): User Commands.
Source: https://docs.oracle.com/cd/E19455-01/806-2902/6jc3b36d5/index.html
0 Response to "List Recursively Files in Big Directory With Continuation Token"
Post a Comment