Pitch modification of audio waveforms

Feb. 13, 1968

W. K. FRENCH ETAL

3,369,077

PITCH MODIFICATION OF AUDIO WAVEFORMS

Filed June 9, 1964

Sheets-Sheet 1

<img file="US3369077A_D0001.tif" />

OJ

TIME

BY

INVENTORS

WALTER K. FRENCH OLIVER W. JOHNSON, JR.

ATTORNEY

Feb. 13, 1968

W. K. FRENCH ETAL 3,369,077

FITCH MODIFICATION OF AUDIO WAVEFORMS

Filed June 9, 1964

Sheets-Sheet 2

<img file="US3369077A_D0002.tif" />

Feb. 13, 1968 w. κ. french etal 3,369,077

PITCH MODIFICATION OF AUDIO WAVEFORMS

Filed June 9, 1964

Sheets-Sheet 3

<img file="US3369077A_D0003.tif" />

Feb. 13, 1968 w. κ. french etal 3,369,077

PITCH MODIFICATION OF AUDIO WAVEFORMS

Filed June 9, 1964

Sheets-Sheet 4

<img file="US3369077A_D0004.tif" />

United States Patent Office 3,369,077

Patented Feb. 13, 1968

3,369,077

PITCH MODIFICATION OF AUDIO WAVEFORMS

Walter K. French, Montrose, and Oliver W. Johnson, Jr., Poughkeepsie, N.Y., assignors to International Business Machines Corporation, New York, N.Y., a corporation of New York

Filed June 9, 1964, Ser. No. 373,751

Claims. (Cl. 179—1)

ABSTRACT OF THE DISCLOSURE

In a speech synthesizing system wherein pitch periods are adjusted according to a predetermined time base, the amplitudes of the beginnings and ends of successive adjusted periods are varied so as to eliminate discontinuities of amplitude in the composite wave form.

This invention relates to the modification of the time duration of an audio waveform and more particularly to the pitch control of an audio signal by adjusting the lengths of the pulse periods of the audio signal waveform and combining the adjusted pulses such that there are no discontinuities between pulses thereby preserving the intelligibility and quality of the information contained in the waveform.

There are many speech and communication systems which utilize prerecorded audio information for playback at desired times or upon random commands. The audio information is prerecorded on magnetic tape or other suitable storage media. If the prerecorded information was of short duration and was continuous such that it could be played back exactly as recorded, the effect on the listener would be the same as if listening directly to the speaker. In the usual case, the recorded information is made up of recordings taken at different times, and also of portions of separate messages pieced together to form a single composite message.

One example of such a system would Ue an automatic stock quotation system which provides an audible message giving the latest stock prices. A given stock price may be formed from a plurality of separate words prerecorded at different times. In such a situation the produced audio signal has been found to be discordant and difficult to understand due to the change in pitch of the separate numbers and the discontinuities in the transition between separate audio signals.

It has also been found that the composite recorded signal can be made more euphonious by adjusting the pitch of each portion of the signal to a standard value and by adjusting the waveforms such that there is a smooth transition between individual segments.

The pitch of a speech sound is determined by the behavior of the vocal cords, which are more accurately described as vocal folds because of the anatomic structure. Whenever a voiced sound is uttered, the vocal folds move together and then apart in such a manner as to vary the size of the opening between them. This opening is referred to as the glottis. For a constant pitch, the vocal folds move together and separate at regular intervals. During a porti jn of each cycle the glottis is completely closed and the supply of air from the lungs causes a rise in pressure which reaches a maximum at this time. When the glottis opens, there is an explosive burst of air which relieves the pressure. The time interval between these bursts determines the fundamental pitch or frequency. The time interval between the pulses, that is, the pitch period, is a reciprocal function of the pitch.

Actually, an acoustical network is interposed between the glottis and the free air. This network serves to modify the nature of the flow by superimposing higher frequencies thereon, but does not modify the pitch. As described, energy flow for producing the voiced sounds comes in 5 explosive bursts and, in a stretching process where the speed of playback is changed, it is inescapable that the time interval between these bursts will be changed. Thus, the resulting pitch is changed proportionally.

It is a characteristic of a periodic wave, no matter how 10 complex that, after a certain time interval known as the period, its form is a repetition of what has gone before. In the case of an exactly periodic wave, the repetition is exact. In the case of a nearly periodic wave (and syllabic rates in speech are so slow compared with voice 15 frequencies of interest that every voiced speech wave is periodic or nearly periodic) the repetition is inexact and approximate, but nevertheless easily recognized. References, hereinafter and in the appended claims, to periodicity of speech is intended to refer to the actual period20 icity of speech; that is, to recognize that speech consists of approximately periodic portions as well as nonperiodic portions, the latter of which are processed by the system as though they were as periodic as the former portions.

Accordingly, it is a primary object of this invention to 25 provide improved apparatus for adjusting the pitch of audio signals.

It is another object of this invention to provide appara_'!s for adjusting the pitch of audio signals on a time domain basis by discriminately subtracting given portions ³θ from the waveform which correspond to pitch periods of the fundamental glottal frequency of the speaker.

Still another object of this invention is to provide apparatus for substracting given portions from the pitch periods of an audio waveform such that the adjusted ³⁹ pitch periods have no discontinuities as a result of such adjustment.

Another object of this invention is to provide apparatus for adjusting the pitch of an audio waveform where the ₄₀ pitch periods of the waveform are to be shortened in accordance with a desired pitch period.

Yet another object of this invention is to provide apparatus for adjusting the length of a pitch period of an audio waveform in accordance with a desired pulse period ₄. wherein the pitch period is shortened by deleting selected portions thereof.

The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of a preferred embodiment 20 of the invention, as illustrated in the accompanying drawings.

In the drawings:

FIG. 1 is a waveform diagram representative of an audio speech signal.

₅₅ FIG. 2 is another waveform diagram representative of an audio speech signal.

FIG. 3 illustrates how FIGS. 3A, 3B, and 3C should be combined.

FIGS. 3A, 3B, and 3C, when combined as shown in 60 FIG. 3, result in a schematic diagram of an embodiment of a pitch control system according to the principles of the present invention.

Referring to FIG. 1, a sample audio waveform is shown plotted along a horizontal axis representative of time and 65 a vertical axis representative of voltage.

The voltage range is adjusted to the range 0 to 64 volts. However, it is desired to raise the “0” plane to the 32 volt level whereby equal variations above and below the “0” plane may be obtained. Thus, the lower unshaded portion ⁷⁰ of the waveform is 0 to 32 volts whereas the upper shaded portion is in excess of 32 volts.

3,369,077

In accordance with the previous description of the fundamental or glottal frequency, and by examination of the vertical lines, a, b, c, etc. on FIG. 1, the fundamental pitch periods are apparent. Thus, the waveform between the lines a and b is almost the same as the waveform between the points b and c and between the points c and d. Also, the length a-b is approximately equal to the length b-c, which is approximately equal to the length c-d, etc. Since the length of the pitch period is representative of pitch, the pitch of the entire waveform is uniform. Likewise, the transition of the waveform from the end of one pitch period to the beginning of the next pitch period is smooth and continuous.

Referring to FIG. 2, a situation is graphically depicted wherein one sound is to follow another sound in a constructed waveform. Two conditions are obvious; first, the pitch period e-f is shorter than the pitch period g-h resulting in a lower pitch sound following a higher pitch sound and second, the waveform level at f is of different amplitude than the waveform level at g resulting in a 20 discontinuity when the waveforms are joined.

These two undesired conditions are overcome by the system to be described. The pitch of the total audio waveform is made uniform by adjusting each pitch period to agree with a desired pitch period. Initially the actual in- 25 put waveform is stored and the beginning and ends of the pitch periods thereof are determined. If the actual pitch period is found to be greater than the desired pitch period, an amount equal to the difference is removed from the end of the actual pitch period.

The pitch periods of the input waveform, now being equal to the desired periods are then joined together. The difference in amplitude between the end of the waveform of one pitch period and the beginning of the waveform of the succeeding pitch period is determined and a D.C. ramp function is formed which, when added to the first waveform, will adjust the amplitude of the end of the first waveform to be equal to the beginning of the suceeding waveform thereby providing a relatively smooth transition.

The first operation to be performed by the system is to determine the beginnings of the pitch periods of the audio waveform signal under consideration. The beginnings of the pitch periods are characterized in that the audio waveform at such point has a maximum positive to negative swing. In the present system the audio waveform is maintained in a storage device and is scanned such that the analog values of the peak to peak transitions are stored. Each peak to peak transition is compared with the next peak to peak transition and the greater of the two is retained for comparison with successive peak to peak transitions until, after a given length of waveform has been scanned, the maximum analog peak to peak transition is retained in a storage circuit and the point of occurrence of such maximum peak to peak transition is specified by a value in a counter.

The maximum peak to peak transitions represent the start of each period of the audio waveform. Each actual pitch period of the audio waveform is then entered into a register and compared in length with a desired pitch period value, whereby the amount to be removed from each actual pitch period is determined. The actual waveforms are shortened by gating them through a gate circuit which opens at the beginning of the actual pitch period and which then closes at a time equal to the desired pitch period. The end of each actual pitch period of the waveform will then abruptly terminate at. some random amplitude. Each the shortened actual pitch periods could then rejoin to form a continuous waveform, however, the amplitudes of the end of one adjusted pitch period and the beginning of the next adjusted pitch period may not be necessarily at the same amplitude, but will be as depicted by f and g in FIG. 2. To make the waveform smooth and continuous, the present system determines the amplitude differences between the beginning and the end of the ad4 justed pitch periods and adds or subtracts a ramp voltage to the end of each adjusted pitch period to raise or lower the amplitude of the end of each adjusted pitch period to be equal with the amplitude of the beginning of the 5 next adjusted pitch period.

An embodiment of a system for carrying out the aforesaid functions is shown in the combined FIGS. 3A, 3B, and 3C. In explaining the operation of FIGS. 3A, 3B, and 3C certain functions are required, such as signal com] θ parisons, signal storage, timing, etc. In the explanation of FIGS. 3A, 3B, and 3C it will be obvious to one skilled in the art that these functions may be carried out by a variety of well known circuit elements. In the interest of simplicity a general purpose computer 3 has been included 15 in the system and will represent the means for carrying out some of the more well known arithmetical and supervisory operations. Any well known general purpose computer having a memory storage and an arithmetic unit under program control will suffice for computer 3.

In FIGS. 3A through 3C audio input 2 provides a source of speech signal. Ganged switch 4 is normally open. When ganged switch 4 is closed, a synchronizing signal is produced by single shot circuit 6 which enters recirculating storage device 8 along with the audio signal from audio input 2. The synchronizing signal and the audio signal are stored on separate tracks. After a predetermined time interval, switch 4 reopens and segment of audio waveform preceded by a synchronizing signal on a separate track continue to circulate in storage device 8.

The continually circulating audio waveform will be selectively employed for various purposes. Initially, the audio waveform from recirculating storage device 8 is applied to a circuit which establishes a time period during which the said audio waveform will be scanned for the presence of a beginning of a pitch period, as manifested by a peak to peak maximum of the audio waveform within the scanned period.

Establishing the glottal period

The period during which the audio signal is scanned for a peak to peak maximum is referred to as the glottal period, that is, a period long enough to insure the occurrence of the beginning of a pitch period (as represented by a peak to peak maximum) yet not too long so as to include ₄₅ the next pitch period beginning. To determine this period the audio signal in delay line 4 is cycled once and applied as an input signal to a high pass filter 10 which removes the low frequency components. The high frequency component signal is then applied to rectifier 12 ₅θ which produces an output signal which is the envelope of the high frequency components. The signal is then passed through a low pass filter 14 which removes the high frequency components, leaving only the envelope of the signal. The envelope of the signal generally resembles _gg a triangular waveform; first increasing and then decreasing. The glottal period is determined by the zero crossings of the positive going portions of the waveform. Thus, the output signal from low pass filter 14 is applied to a zero axis crossing detector 16 which generates an ₆θ output signal only upon the occurrence of positive going zero axis crossings of the envelope. The output of zero axis crossing detector 16 is applied to a counter 18 which will begin to count upon the occurrence of the first positive going zero axis crossing and stop upon the oc_Gg currence of the next positive going zero axis crossing.

A period is therefore established in counter 18 from zero to a count which what will be referred to as T_E. This period is an approximate representation of the glottal period. Another operation which occurs during the first cycle of the audio waveform in storage device 8 is that the synchronizing signal in the storage device 8 is applied, through a synchronizing signal detector 20 to counter 22. Counter 22 is also connected to a continually running oscillator 24 which causes the counter 22 to continually 75 count. The synchronizing signal from synchronizing de-.

3,369,077 tector 20 resets the counter 22 so that a count from zero begins immediately after the occurrence of the synchronizing signal at each cycle of storage device 8. Sometime after the occurrence of the synchronizing signal a voiced audio signal will occur. The present invention relates to 5 pitch adjustment, and pitch is a characteristic of voiced rather than fricative sounds. Therefore, the output of storage device 8 is applied to a voicing detector 26 which produces an output signal upon the occurrence of voiced audio signals from storage device 8. The signal from voic- , „ ing detector 26 is applied to a gate circuit 28 which gates the count of counter 22 at the time of the voiced sound into a register 30. The count in counter 22 at the beginning of the voiced sound will be referred to as T_v. The signal from voicing detector 26 at count T_v is also applied 15 to a gate circuit 32 at the output of counter 18 to gate the glottal period count T_g into an add circuit 34 where it is added to the T_v count from counter 22 and stored in register 36. Thus, during the first cycle of the audio waveform in delay element 4 the following steps are per- 20 formed. First, an approximate glottal period T_g is established in the form of a count in counter 18, counter 22 is set to zero by the synchronizing signal in storage device 8 and the later count Ty at the time of the beginning of voiced signals is read into register 30 and a count equal 25 to the beginning of voiced signals T_v plus the count T_gfrom counter 18 is read into register 36. Thus, registers 30 and 36 contain counts which differ in the amount equal to the glottal period count T_g. It is to be noted that the output of counter 22 is connected to a first comparator 30 circuit 38 which is in turn connected to register 30 and to a second comparator circuit 40 which is in turn connected to register 36. Thus, the count of counter 22 may be compared with the counts set in both register 30 and register 36. 35

Determining the pitch periods

Upon the second cycle of storage device 8 the system is ready to determine the beginning of a pitch period by determining the time of occurrence (via a count in counter 22) of the maximum peak to peak difference of ^4,0the audio waveform signal in the period between the beginning of the voiced audio signal T_v as set in register 30 and the time count T_v+T_g as set forth in register 36.

During the second cycle of the storage device 8 the second occurrence of the synchronizing signal resets the 45 counter 22 to zero and oscillator 24 begins the new count. When counter 22 reaches the count T_v, a match occurs with the contents of register 30 through comparator 38 causing an output signal from comparator 38 which is coupled to a flip-flop circuit 42. An output signal thereby 50 results from the “1” side of flip-flop 42 which will be referred to as the Q signal.

The Q signal from flip-flop 42 is applied directly to a gate circuit 44. The output from voicing detector 26 is also applied to gate circuit 44, thus gate circuit 44 opens 55 (on the second cycle) at a time count of T_v. The Q signal from flip-flop 42 is also applied to analog hold circuits 46, 48, 50, and 52 (to be later described) to clear or erase any analog signals therein. The output of flip-flop 42 is also applied to a circuit 54 for detecting a negative going go signal change (for example, a differentiator in series with a reversed diode) to produce a signal at the end of the Q pulse to be later explained.

Gate circuit 44 being opened, the audio waveform from storage device 8 is applied to analog gates 56, 58, 60, and ¢5 62, positive peak detector 64 and negative peak detector 66.

Positive peak detector 64 produces an output signal referred to hereinafter as +p in response to each of the positive peaks of the audio waveform and negative peak 70 detector 13 produces an output signal hereinafter referred to as — p in response to each of the negative peaks of the audio signal. The employment of the -j-p and —p signals will be later described.

The output of recirculating storage device 8; i.e., the 75 speech signal segment, is conducted through gate 44, and as previously stated, is applied to analog gates 56, 58, 60, and 62. As will be described in detail, the gate circuits 56 and 58 are employed to gate one pair of positive and negative analog waveform peaks into analog hold circuits 46 and 48 respectively, and gate circuits 60 and 62 are used to gate another pair of positive and negative analog peaks into analog hold circuits 50 and 52 respectively. The registers 68 and 70 store the times (in the form of a count) at which the positive peaks occur. The amplitudes of the peak to peak analog voltages (that is, the difference between the positive and negative peaks) are compared by other circuitry.

The output signal from synchronizing detector 20’ is also applied to a flip-flop circuit 72 to set the “1” output of flip-flop 72 to the high level state upon the occurrence of the synchronizing signal. The output signal on the “1” bit output lead of flip-flop 72 will be hereinafter referred to as signal la.

The la signal from flip-flop 72 is applied, among other places, to analog gate 56. Thus, upon the occurrence of the first positive peak of the audio signal stored in storage device 8 following the synchronizing signal, the +p signal from peak detector 64 being applied to gate circuit 56, will open gate circuit 56 and permit the analog value of the positive peak to be stored in analog hold circuit 46,

The la signal from flip-flop 72 and the -j-p signal from peak detector 64 are also applied to gate circuit 74 between counter 22 and register 68 so that the occurrence time (in the form of a count) of the -f~p signal is stored in register 68. The la signal from flip-flop 72 is further applied to OR circuit 76, the output of which is applied to AND” circuit 78 along with the -|-p signal from positive peak detector 64 to provide an output signal from AND” circuit 78 to set a flip-flop circuit 80. When flipcircuit 80 is set, an output signal is provided on the 1 bit output lead 82 which is in turn applied to delay circuit 84, “AND” circuit 86, and “AND” circuit 88. Signa! la from flip-flop 72 is also applied to “AND” circuit 86 which therefore produces an output signal hereinafter referred to as signal lb. The signal lb from “AND” circuit 86 is applied to analog gate circuit 58 so that upon the occurrence of the following negative peak of the audio signal the resultant negative peak signal __p f_romnegative peak detector 66 gates the analog value of the negative peak to analog hold circuit 48 and is then compared with the analog of the positive peak in analog hold circuit 44 at differential amplifier 90.

The output of differential amplifier 90 is the peak to peak value of the first positive to negative excursion of the input audio voiced signal. The output signal from differential amplifier 90 is inverted by inverter circuit 92 and is applied to analog add circuit 94. Circuits 46 and 48 being analog hold circuits, the signal applied to add circuit 94 is maintained until such time as a new signal is applied through gate 56 and/or 58. At this point in the discussion there is no output as yet from the differential amplifier 96 since the necessary gating signals 2a and 2b for gates 60 and 62 (to be later described) have not yet been generated. Therefore, analog hold circuits 50 and 52 are still at their initial zero level. Thus, the only input to ^{rcuit 94 1S the ne}g^ative signal from inverter circuit 92. The output signal from add circuit 94 is therefore also negative and will be conducted through a reversed diode 96 to a gate circuit 98.

Gate circuit 98 is gated by the la signal from flip-flop 72 and the signal from analog add circuit 94 passing through diode 96 is gated through to gate circuit 100 where it is gated by the —p signal from negative peak detector 66 which has been slightly delayed in delay circuit 102 in order that the analog waveform values have had time to become established in the analog hold circuits 46 and 48. The signal gated through “AND” circuit 100 is applied to “AND” switches flip-flop 72. The switching of

3,369.077 flip-flop 72 results in the “1” output thereof going low (signal la thereby ceasing) and the “0” output thereof going high, producing an output signal therefrom referred to as signal 2α.

In the meantime the lb signal from “AND” circuit 86 ceases since the signal on lead 82 of flip-flop 80 is fed back to reset flip-flop 80 through a delay circuit 84 (which provides a delay sufficient to allow the analog value in hold circuit 48 to be established).

The la and lb signals no longer being present, gate circuits 56 and 58 will remain closed as well as gate circuit 74, during the occurrence of the next positive and negative peak signals of the voiced analog w'aveform. The next positive peak of the analog waveform will produce another +p signal from positive peak detector 64 which is applied to gate circuits 56, 58, 60, and 62 and gate circuits 74 and 104. The gate circuit 104 is located between counter 22 and register 70. The 2α signal from flipflop 72 is now present so that gate circuits 60 and 104 are opened. The opening of gate circuit 60 allows the analog positive peak of the audio waveform to enter analog hold circuit 50 and the opening of gate circuit 104 permits the time of occurrence count of this positive peak to be stored in register 70.

The 2α signal from flip-flop 72 is also applied to “OR” circuit 76, the output of which is gated with the +p signal at “AND” circuit 78 to switch flip-flop 80 which in turn will again provide an output signal on lead 82. The output signal on lead 82 will be gated with signal 2α at “AND” circuit 88 to provide an output therefrom which will be referred to as signal 2b. A lb output signal will not be produced by “AND” circuit 86 because the necessary la signal is not present at the input thereof. Signal 2b is applied to gate circuit 62 along with the output of negative peak detector 66 so that upon the occurrence of the next (second) negative peak of audio waveform, a —p signal from peak detector 66 is produced which will gate the negative peak analog signal into hold circuit 52.

The 2b signal also ceases by the action of the signal on lead 82 of flip-flop 80 being fed back through delay circuit 84 to reset flip-flop 80, thereby disabling “AND” circuit 88. The delay circuit 84 again provides a delay sufficient to allow the analog value to be established in hold circuit 52. Hold circuit 50 (containing the second positive peak analog signal) and hold circuit 52 (containing the second negative peak analog signal) are coupled to a differential amplifier 96 which provides an output signal equal to the peak to peak value of the second positive and negative waveform peak signals. The output signal from differential amplifier 96 is applied to add circuit 94 where it is compared with the inverted peak to peak analog value of the first positive and negative peaks of the audio waveform from differential amplifier 90 (via inverter 92). The output signal from inverter circuit 92 will either be greater than the output signal from differential amplifier 96 in which case the output signal from add circuit 94 will be negative, or will be less than the output signal from amplifier 96, in which case the output signal from add circuit 94 will be positive. If the output signal from add circuit 94 is negative, it means that the peak to peak value of the first positive to negative transition of the audio waveform (the output of differential amplifier 98) is greater than the peak to peak value of the second positive to negative transition of the audio waveform (the output of differential amplifier 96). This means that the first and second peak signals may represent the beginning of a pulse period and that the second peak to peak signals do not. In such instance it is desired to continue to hold the values in hold circuits 46 and 48 and to store the third positive and negative peaks of the audio signal in hold circuit 50 and 52 (in place of the second positive and negative peaks) and compare the difference (output of amplifier 96) with the first positive to first positive to negative peak transition is exceeded or until the end of the sampling period.

If the second positive to negative peak transition does exceed the first (i.e., the output of add circuit 94 being 5 positive), the initial peak to peak transition of the audio waveform is smaller than the second peak to peak transition and is not representative of the beginning of the pitch period. Thus, the second peak values are held in hold circuits 50 and 52 and the third positive and negative peak values of the audio signal are stored in hold circuits 46 and 48 for comparison therewith.

The general operation of this portion of the system is therefore to store the first peak to peak analog value of the audio waveform in a first hold channel and compare 15 such value with subsequently occurring peak to peak analog values which are entered into a second hold channel. The initial peak to peak value in the first hold channel will remain stored until a peak to peak value in the second hold channel exceeds it in value, at which time 20 the new maximum peak to peak value remains stored in the second hold channel and further subsequently occurring peak to peak values are entered in the first hold channel (replacing the initial peak to peak values previously stored) for continued comparison. Thus, at the end 25 of the glottal period as set forth by the values in registers 30 and 36, a maximum peak to peak transition of the audio waveform will be determined which will in turn determine the beginning of the pitch period of the sound.

For example, if the second peak to peak transition were 30 greater than the initial transition, the output from add circuit 94 will be positive and pass through diode 106 and be applied to gate circuit 108 where it will be gated by the still present 2α signal (note, if the output from add cricuit 94 were negative it would indicate that the initial peak 35 to peak transition was larger and such signal would pass through diode 96 to gate 98 which does not become gated since signal la is absent). The signal gated through gate 108 is applied to gate circuit 100 where it is gated by the —p signal produced by the second negative peak signal ⁴⁰ which has been delayed by delay circuit 102. Delay circuit 102 provides a delay sufficient to allow the values in the analog hold circuits 46, 48, 50, and 52 to be established. The signal gated through gate circuit 100 is applied to flip-flop 72 to switch it to its original state, that is, to 45 provide the la signal so that the next (third) positive and negative analog peak signals are stored in hold circuits 46 and 48. If, however, the initial peak to peak transition had been larger than the second transition and the output of add circuit were consequently negative, then 5θ it can be seen that no such negative signal would not be passed through gate 41 due to the absence of the la signal and no switching signal would be applied to flipflop 72. Signal 2a would, therefore, remain present. Thus, the third positive and negative analog peak signals would 55 be entered into hold circuits 50 and 52 for comparison with the initial peak to peak transition.

In like manner each subsequent peak to peak transition of the voiced audio waveform will continue to be compared with the previously stored most maximum peak to 60 peak transition and if greater, will displace the previous stored maximum value in the analog hold circuits and will itself be compared with further occurring peak to peak transitions. Counter 22 will continue to run until it reaches the value T_v-j-T_g, which is the value stored in 65 register 36. At this time a compare signal is generated by comparator circuit 40 which is applied to and switches flip-flop 42, causing the Q output signal therefrom to drop. The dropping of the Q signal results in gate 44 closing a no further audio waveform being applied to gates 56, 70 58, 60, or 62.

The “1” output of flip-flop 42 is also applied to circuit 54 for detecting the decreasing or lagging edge of the Q signal. Circuit 54 may, for example, include a differentiating circuit and a reversed diode. The signal is negative peak difference, and so on until the value of the produced from circuit only at the end of the Q pulse.

3,369 9

The signal from circuit 54 is used to read out the time count at which the maximum peak to peak transition of the audio waveform occurred. There will be a count in register 68 and a count in register 70, one of which represents the count of the time of the maximum peak to 5 peak transition of the audio waveform and therefore the beginning of a pitch period.

If flip-flop 72 is in its first state such that signal la is present, then the count in register 68 represents the pitch period beginning, but if flip-flop 72 is in its second state ₁₀such that signal 2α is present, then the count in register 70 represents the pitch period beginning. Thus, the signal from circuit 54 is connected along with the “1” output lead of flip-flop 72 to an “AND” circuit 110, the output of which is coupled as a read-out signal to register 68. 15 The “0” output lead of flip-flop 72 is connected along with the output of circuit 54 to an “AND” circuit 112, the output of which is coupled as a read-out signal to register 70. At the end of the Q pulse therefore, if the la signal is present, register 68 is read out, and if the 2α 20 signal is present, register 70 is read out.

_ Whichever register is read out, a count will be provided which represents the time of occurrence of the first pitch period beginning thus far detected.

This count will be referred to as T, and will fall some- 25 where between the counts T_v and T_v+T_s.

As previously stated, at the end of the Q pulse (time 7_v+7_g) the Tj count is read out of either register 63 or register 70. This T, count is applied to and stored in the computer 3 via leads 114 or 116, where a computa- 30 tion is performed such that to the T, count is added onehalf the glottal period count T„ to produce the value Ti-pATc.. A second computation is performed wherein one and one-half the glottal period T_g is added to T, to produce the value Ti+^Tg. These values having been 35 completed, they are applied from the computer 3 to registers 30 and 36 via leads 118 and 120 respectively with the values T^VzT, being stored in register 30 and the value 7,+%7_g being stored in register 36.

What has occurred has been the selection (in the form ⁴⁰of a count) of the beginning of a pitch period within the audio waveform as the stored audio waveform was cycled once in storage device 8. The location of the beginning of the pitch period T, was, by predesign, located in the glottal period between Tv and Tv+Tg. With the begin- 45 ning of this pitch period at T, established, the beginning of the next pitch period will be located, and will be looked for within the period between 71+¹/27g (stored in register 30) and Ti+%T_g (stored in register 36) because the beginning of the second pitch period will be 50 located approximately in the middle of this period.

Nothing further happens in the system until the cycle of the storage device 8 ends and a new cycle begins. At the beginning of the new cycle of storage device 8 the synchronizing pulse preceding the audio waveform resets 55 counter 22 and sets flip-flop 72 such that a la signal is present on the “1” output lead thereof. As before, the la signal is applied to gate circuits 56 and 74.

The counter 22 will continue to count until a value Tj+’ATg is reached, this being the count stored in reg- 50 ister 30. At this time a compare signal is generated by comparator circuit 38 which switches flip-flop 42 such that a Q signal again appears on the “1” output lead thereof. The Q signal from flip-flop 42 is applied to and clears analog hold circuits 46, 48, 50, and 52 and is applied to 65 and opens gate circuit 44. The opening of gate circuit 44 permits the voiced audio waveform from storage device 8 to pass therethrough and be applied to gate circuits 56, 58, 60, and 62, and peak detectors 64 and 66. This, as previously stated, occurs at a time count of 71+½ 7_g, 70 The next positive peak thereafter occurring will produce a +p output signal from peak detector 64 which is applied to gate 56 and, the la signal being present, gate 56 is opened and the analog value of the positive peak is stored in hold circuit 46. « ,077

As described for the previous cycle, the la signal also passes through “OR” circuit 76 and is gated by the -|-p signal through “AND” circuit 78 to switch flip-flop 80. A signal is thereby produced on lead 82 which is gated by the la signal at “AND” circuit 86 to produce a lb signal at the output thereof. The lb signal is applied to gate circuit 58. The next negative peak of the audio waveform will generate a —p signal from peak detector 66 which will be applied to and open gate circuit 58 so that the analog value of the negative peak will pass therethrough and be stored in analog hold circuit 48. The peak to peak value of the audio waveform is then produced by differential amplifier 90.

The output signal from differential amplifier 90 is inverted by inverter circuit 92 and applied to analog add circuit 94 where it is compared with the output signal from differential amplifier 96 (which is zero at this time). Thus, the output signal from add circuit 94 is negative and will pass through diode 96 and be applied to gate circuit 98.

Gate circuit 98 is gated by the la signal from flip-flop 72 and the signal from analog add circuit 94 passing through diode 96 is gated through to gate circuit 100 where it is gated by the —p signal from peak detector 66 which has been slightly delayed in delay circuit 102 in order that the analog waveform values have had time to become established in the analog hold circuits 46 and 48. The signal gated through gate circuit 100 is applied to and switches flip-flop 72. The switching of flip-flop 72 results in the “1” output thereof going low (signal la thereby ceasing) and the “0” output thereof going high, producing a 2α output signal therefrom.

In the meantime the lb signal from “AND” circuit 86 ceases since the signal on lead 82 of flip-flop 80 is fed back to reset flip-flop 80 through the delay circuit 84.

The la and lb signals no longer being present, gate circuits 56 and 58 will remain closed as well as gate circuit 74, during the occurrence of the next positive and negative peak signals of the voiced analog waveform. The next positive peak of the analog waveform will produce another -j-p signal from positive peak detector 64 which is applied to gate circuits 56, 58, 60, and 62 and gate circuits 74 and 104. The 2α signal from flip-flop 72 is now present so that gate circuits 60 and 104 are opened. The opening of gate circuit 6® allows the analog positive peak of the audio waveform to enter analog hold circuit 50, and the opening of gate circuit 104 permits the time of occurrence count of this positive peak to be stored in register 70. The 2α signal from flip-flop 72 is also passed through “OR” circuit 76, gated through “AND” circuit 78, switches flip-flop 80 thereby resulting in a signal on lead 82. The signal on lead 82 is gated through “AND” circuit 88 by the 2α signal from flip-flop 72 and a 2b output signal is produced at the output of “AND” circuit 88. The 2b signal is applied to analog gate circuit 62 so that when the next negative peak of the audio waveform, produces a —p signal from negative peak detector circuit 66, gate circuit 62 opens and the analog value of the negative peak of the audio waveform will be stored in analog hold circuit 52. The difference between the positive and negative peaks stored respectively in hold circuits 50 and 52 is produced by differential amplifier 96. The output signal from differential amplifier 96 is applied to add circuit 94 where it is added with the inverted negative signal from differential amplifier 90.

The circuit then operates as previously described, that is, the largest peak to peak difference is retained in the hold circuits and compared with successive peak to peak values until a larger value is found. The only difference in the present cycle and the previous cycle is that the audio waveform is scanned between the time counts of 7i+^I/47_g and 7!-|-^7g whereas in the previous cycle the audio waveform was scanned between the time counts of 7_V and 7_v+7_g.

3,369,077

When counter 22 reaches the time count 7ί+%7_Βit compares with the count in register 36 and a compare signal is generated by comparator circuit 48 which switches flip-flop 42 such that the Q signal therefrom closes.

The end of the Q signal causes gate circuit 44 to close and a signal to be produced by negative going signal detector circuit 54 which is gated with the la signal or the 2α signal (whichever present) at “AND” circuits 110 and 112 and will read out the count in the proper one of registers 68 and 70. The count thus read out will represent the time of occurrence of the positive peak of the largest peak to peak audio waveform value in the period from Ti+¹A7g to 7i+%7g, and consequently will be the time of occurrence of the beginning of the second pitch period. This time count will be referred to as T2. The T2 time count read out of either register 68 or 70 is applied to and stored in the computer 3 via the associated ones of leads 114 and 116, where a computation is performed such that to the T2 count is added one-half the glottal period count Tg to produce the value 72+¹/iTg. A second computation is performed wherein one and one-half the glottal period Tl; is added to T₂ to produce the value T₂-j-%T_g. These values are then transmitted from the computer to registers 57 and 58; register 57 having the 7₂+ ΆΤ, value stored therein and register 58 having the 7₂+%7_g value stored therein.

In a manner identical to that previously described, on the next cycle of storage device 8 the voiced audio waveform is scanned in the period between 72-(-½7_g and 7₂+%7_g for a maximum peak to peak value which represents the beginning of the next pitch period. Such next pitch period beginning time count, referred to as T₃, is stored in the computer 3 and a new scan period 7₃+^JA7_g to 7₃+%7_g, is established within which the next pitch period beginning T₄ is determined. Ultimately, the entire waveform stored in storage device 8 will be scanned and a series of time counts T_b T₂, T₃, T₄, . . . T_n representng the beginnings of all of the pitch periods are stored in the computer 3.

The time distance between each of the pitch period beginnings are the actual pitch periods which determine the pitch of the sounds. The various pitch periods, that is, T₄ to T₂, T₂ to T₃, T₃ to T₄, T₄ to T₅, etc. will differ in length. As previously stated, these pitch periods will be adjusted to a uniform length equal to a desired pitch period referred to as T_d.

Establishing desired pitch periods

During this phase of system operation a given amount of waveform from each pitch period will be removed or cut-off to shorten each pitch period to agree with the desired, pitch period T_d. Due to this end of period waveform cut-off, the end of one pitch period waveform and the beginning of the next pitch period may be discontinuous. For example, the amplitude levels may differ as illustrated by points / and g in FIG. 2. Thus, in addition to shortening given pitch periods to agree with T_d, the amplitude of the ends of waveforms may have to be raised or lowered to the level of the beginning of the next successive pitch period waveform.

As a practical matter, the value of the desired pitch period T_d may be selected as equal to the smallest of the actual pitch periods Ti to T₂, T₂ to T₃, T₃ to T₄, etc. This will insure that each of the actual pitch periods will be either equal to or greater than the desired pitch period.

In the following description all of the pitch periods will be adjusted to conform to a single desired pitch period T_d, however, it might also be possible that the different pitch periods might be adjusted to conform to different desired pitch periods; for example, one desired pitch period may be employed for open vowel sounds and another desired pitch period may be provided for closed vowel sounds. The use of more than one desired pitch period would require minor modifications in the system of FIG. 3.

As will be seen from the following discussion, the recirculating storage device 8 continues to cycle after 5 the determination of all the pitch period beginning T, through T_n. On the first cycle after the determination of T_n the amplitude difference between the end of the first desired pitch period at 7i+?_d and the beginning of the next pitch period at T₂ is determined. On the second IQ cycle of storage device 8 after the determination of T_n, a ramp function of the amplitude difference is added to the first desired pitch period waveform, which is then stored in computer 3. On the third cycle of storage device 8 after the determination of T_n the amplitude difference 15 between the end of the second desired pitch period at 7₂+7_d and the beginning of the third pitch period at T₃is determined. On the fourth cycle after the determination of T_n this amplitude difference is added to the second desired pitch period waveform and the second adjusted 20 desired pitch period waveform is stored in computer 3. In like manner the rest of the adjusted desired pitch period waveforms are constructed on successive even cycles and stored in computer 3.

A more detailed discussion of this operation now 25 follows.

On the next cycle of the storage device 8 after the determination of the final (T_n) pitch period beginning, the counter 22 is again reset to zero by the synchronizing pulse from storage device 8. It is to be noted that the 30 output from recirculating storage device 8 is applied to a gate circuit 122 which is controlled by a flip-flop circuit 124. When the flip-flop 124 is in its “1” state gate 122 is opened. Flip-flop 124 is switched by a signal from computer 3. Hie flip-flop 124 is set to its “1” state 35 by a signal on lead 128 and to its “0” state by the signal on lead 126 delayed an amount T_d by delay circuit 128. The output of counter 22 is fed into computer 3 for comparison with the pitch period beginning counts T₁₍T₂, T₃, T₄, . . . T_n. When the counter 22 reaches the 40 count T₄, it compares with the count T_1; stored in the computer 3 and a pulsed signal is produced from the computer 3 on lead 126 to set flip-flop 124 to its “1” state, thereby opening gate 122 and permitting the audio waveform from storage device 8 to pass therethrough. The pulse on lead 126 is delayed an amount T_d by delay circuit 128, thus the flip-flop 124 is switched to its “0” state at a time Ti+T_d- When flip-flop 124 is switched to its “0” state gate 122 is closed. It is seen that gate circuit 122 was open for the period Tj through ?i+T_d. The 50 audio waveform was passed therethrough for this period so that a portion of audio waveform is produced at the output of gate circuit 122 which begins at the pitch period beginning T_t and continues for the desired pitch period T_d. This waveform portion, which has the desired pitch 55 period T_d, is applied to an add circuit 130, which is in turn connected to the input of an analog to digital converter 132. During this first cycle of the storage device 8 (from Ti to 71+7/.) and each successive odd cycle the output from analog to digital converter 132 is in00 hibited by a signal from computer 3. The pulse from delay circuit 128 occurring at time Ti+?_d is also applied to an analog gate circuit 134 which gates the analog value of the audio waveform at the time Tj+Ta to an analog hold circuit 136. A short time after time 7i+7_d the β5 counter 22 will reach the count T₂. The count T₂ compares with the T₂ value stored in computer 3 and a pulse is produced on computer output lead 138 to momentarily open analog gate 140, which gates the analog value of the audio waveform to analog hold crcuit 142. The analog 70 value of the audio waveform at time 7+-7,. (the end of the first desired pitch period) stored in hold circuit 136, and the analog value of the audio waveform at time T₂ (the beginning of the next pitch period) stored in hold circuit 142 are compared in a different amplifier 75 144, the output of which is the difference between the

3,369.077 two analog values. The output of differential amplifier therefore is the 'amplitude difference between the audio waveform at the end of the first desired pitch period and the beginning of the second pitch period. This difference voltage is referred to as V_d.

_ The difference voltage V_d is applied to an integrating circuit 146 consisting of an input resistor 148, an operational amplifier 150 with feedback capacitor 1'52, the output of which is connected to ground potential through a normally closed switch 154. Switch 154 is connected to a solenoid 156 which is in turn actuated the pulsed signals on leads 126 and the pulsed signals T_d occurring later from delay circuit 128. Thus, switch 154 is normally closed, is opening at timer T_ls and closes again at time

Ί\+Τ_α opens again at T₂ and closes at time T₂+T_d, etc. The resultant output of integrating circuit 146 is a ramp voltage which begins at zero at the beginning of each pitch period Tj, T₂, T₃, etc. and which builds up to the associated value of voltage V_d at a time T_d later. During this presently described cycle what has occurred is that a segment audio waveform has been passed through gate circuit 122 betwen the times T₄ and Τι4~Τ&lt;ι· This segment of waveform is applied through add circuit 130 to the digital to analog converter 132 which is inhibited for this first present cycle (and for all successive odd number cycles). At time T^+T^ the analog value of the amplitude of the audio waveform at this time is stored in hold circuit 136 and at time T₂ the analog value of the audio waveform at this time is stored in hold circuit 142. The difference between the two amplitudes (V_d) is determined and applied to integrating circuit 146.

The circulating storage device 8 completes its cycle with nothing further occurring. On the next (second) cycle of the storage device 8 counter 22 is reset and recounts. At time T, the signal from computer 3 on lead 126 switches flip-flop 124 thereby opening gate 122 allowing the audio waveform to pass therethrough. At the same time switch 154 of integrator circuit 146 is opened so that the ramp voltage output thereof begins to build. The output of integrator circuit 146 is applied to add circuit 130 and with the audio waveform from gate 122. The audio waveform is applied to add circuit 130 for the period Tj to Zi+7_d. During this period the output signal from integrator circuit 146 increases from zero at T₄ to V_d at Ti+T_d. Thus, the output signal from add circuit 132 is the audio waveform with a desired pitch period at T_d and and amplitude at time 7i+T_d increased by an amount V_dso that it is the same amplitude as the beginning of the next pitch period at time T₂. This being an odd cycle of the storage device 8, the inhibit signal at the output of the analog to digital converter 132 is removed and an output signal is produced therefrom which is the digital equivalent of the audio waveform during the first desired pitch period T₄ to T_r+T_&amp; with the voltage adjustment V_dadded thereto. This digital output signal is transmitted to computer 3.

On the next cycle of storage device 8 the second desired pitch period waveform is formed and the amplitude of .the end thereof is adjusted to equal the beginning of the third pitch period. When counter 22 reaches t₂ a pulse is generated by the comparison with the value T₂ in computer 3 and this pulse is transmitted on lead 126 to switch flip-flop 124 to its “1 state. The flip-flop 124 opens gate 122 and permits the audio waveform to pass through until time T₂+T_d, when the delayed pulse through delay circuit 128 switches flip-flop 124 to its “0” state and gate 122 closes. Thus, a second waveform segment beginning at time T₂ and ending at a period T_d later is passed through gate 122 to add circuit 130 and to analog to digital converter 132. Analog to digital converter is inhibited on this cycle and no output signal therefrom is produced. At time T₂+T_d the output pulse from delay circuit 128 open analog gate circuit 134 entering the analog value of the audio waveform at time T₂+T_d into hold circuit 136. A short time after the occurrence of time T₂+T_d.counter 22 reaches the count T₃ value and compares with the T₃ value in computer 3. A pulse is then 5 generated on lead 138 which opens gate 140, allowing the analog value of the audio waveform at time T₃ to enter hold circuit 142. The amplitude of the audio waveform at the end of the second desired pitch period (T₂+T_d) is then compared with the amplitude of the 10 audio waveform at the beginning of the third pitch period T₃ by means of differential amplifier 144 is another difference voltage V_d and is applied to integrator 146.

The present cycle of the recirculating storage device continues with no further effect and at the beginning of 15 the next cycle counter 22 is reset. At time T₂ of this next cycle gate 122 is opened and the segment of audio waveform signal is passed through to add circuit 130. Also at this time T₂ switch 154 of integrator 146 is opened and the output of integrator 146 begins to increase in volt20 age. The increasing voltage from integrator circuit 146 is added to the audio waveform signal at add circuit 130 until time T₂+T_d, at which time the value added to the audio waveform signal is V_d and at which time gate 122 is closed and switch 154 is closed. The output from ana25 log to digital converter 132 is not inhibited at this cycle so that a digital representation of the second desired pitch period with adjusted amplitude is produced as an output signal and transmitted to computer 3.

In like manner on successive odd cycles the remainder 30 of the audio waveform is gated through gate 122 between tile beginning of each of the rest of the pitch period beginnings T₃, T₄, . . . T_n for a time period equal to T_dsuch that the voltage difference between the end of one desired pitch period and the beginning of next pitch 35 period (i.e., voltage V_d) may be calculated. On successive even cycles the amplitude differences (V_d) between the waveforms at the end of each pitch period and the beginning of the next pitch period so determined are added to the audio waveform signal so each pitch period is ad40 justed by the addition of a voltage which results in the ends of the pitch periods and the beginnings of the next pitch periods being at the same amplitude. The adjusted desired pitch periods so produced on the successive even cycles pre converted to digital representation by analog 4.k to digital converter 132 and are transmitted to computer 3.

The digital inputs to the computer 3 from analog to digital converter 132 is the audio waveform having pitch periods of desired length and adjusted so the end of each 50 pitch period is at the same voltage amplitude as the beginning of the next pitch period. As previously stated, these pitch periods are constructed on every other cycle of the recirculating storage device during this phase of the operation. Thus, the pitch period input signals to 55 computer 3 are spaced in time and form a continuous waveform. The butting or joining together of the separate pitch periods in digital form may take place in computer 3 which has the suitable storage and switching units for this purpose.

Thus, it is seen that a complete audio signal is stored in computer 3 m digital form. Each of the pitch periods stored in digit al. form in the computer have pitch periods equal to the desired pitch period T_d and do not have any discontinuities between the ends and beginnings of suc05 cessive pitch periods. The stored audio signal may be read out of the computer 3 and stored as an audio signal on magnetic tape for practical use.

What has been described is a system wherein segments of different audio waveforms may be combined together 70 to form a single audio waveform or message. In order to overcome the pitch changes between segments, each pitch period of the composite audio waveform is made to conform or be equal to a selected standard pitch period. At the same time the amplitude differences between 75 the ends and the beginnings of the separate pitch periods

3,369,077 ¹⁶ with a desired pitch period length and adjusting each pitch period to conform with the desired pitch period length, means for determining the amplitude of the beginning and end of each of said length adjusted pitch periods, and means for adjusting the amplitude of the end of each of said length adjusted pitch periods to equal the amplitude of the beginning of each successive ₁₀ length adjusted pitch period.

6. A system according to claim 5 wherein said means for analyzing said data signal for determining maximum peak to peak amplitude differences within selected regions of said data signal includes:

means for storing and comparing the peak to peak amplitude differences within selected successive regions of said data signal for determining the maximum peak to peak amplitude difference within each of said successive selected regions, counter means synchronized with said storing and comparing means for providing a count representative of each positive peak associated with said maximum peak to peak amplitude differences with said selected regions, means for establishing a plurality of equal length regions within said data signal, each equal length region beginning with each of said positive peaks associated with said maximum peak to peak differences and ending at a position prior to the next positive peak of the next successive maximum peak to peak transition, means for processing said equal length regions of said data signal for producing a separate difference signal representative of the amplitude difference of the data signal between the end of each equal length region and the beginning of the next successive equal length region, means for adjusting the amplitude of each equal length region by applying the difference signal thereto de40 termined by the amplitude of the data signal at the end of said region and the beginning of the next equal length region, and means for joining the end of each adjusted equal length regions to the beginning of each successive adjusted equal length region to form a continuous data signal composed of equal length regions.

7. A system according to claim 6 wherein said means for storing and comparing includes a circulating storage device having a predetermined amount of audio wave50 form signal stored therein, said audio waveform signal being periodically repetitive, means for storing the analog value of each positive peak to negative peak transition of said audio waveform signal, and means for comparing the analog value of each peak to peak transition with successive peak to peak transitions within selected successive regions of said total waveform signal and storing the location of the positive peak of the maximum peak to peak transi60 tion within each of said selected regions, and wherein said counter means is synchronized with said circulating storage device for providing a count representative of the location of each of said positive peaks of the maximum peak to peak transition within each of said selected regions.

8. A system for modifying audio data signals comprising:

means for combining discrete increments of audio data into a composite audio waveform signal, means for applying said audio waveform signal to a circulating storage device, said audio' waveform signal stored therein being periodically repetitive, means for storing the analog value of each positive peak to negative peak transition of said audio wave75 form signal,

1g are compensated for by the addition of calculated ramp voltages. The resultant audio waveform will be of uniform pitch and have no discontinuities and will sound normal to the human ear.

While the invention has been particularly shown and described with reference to a preferred embodiment thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.