8: A Python String Pipe, Which Conveys Any String Serially

<The previous article in this series | The table of contents of this series | The next article in this series>

Serially because otherwise, the memory would be used up. A Python equivalent for Java 'java.io.PipedWriter' + 'java.io.PipedReader'.

---

Topics

About: The Python programming language

---

The table of contents of this article

• Starting Context
• Target Context
• Orientation
• Main Body
• 1: The Whole of a Large String Cannot Be Loaded into the Memory
• 2: Java Has 'java.io.PipedWriter' + 'java.io.PipedReader'
• 3: Python Does Not . . .
• 4: The Plan
• 5: The Code
• 6: An Example Usage and an Execution Result

---

Starting Context

• The reader has a basic knowledge on the Python programming language.

---

Target Context

• The reader will know a string pipe, which conveys any string serially.

---

Orientation

Here is introduced a sting pipe, which conveys any string serially, which is necessary in order to prevent the memory from being used up, without deforming the code structure.

The string pipe is a subclass of the objects pipe introduced in the previous article.

There will be also the Java version, the C++ version, and the C# version.

---

Main Body

---

1: The Whole of a Large String Cannot Be Loaded into the Memory

Hypothesizer 7
If a string is small, it can be just loaded into the memory as a string object.

If it is large enough, that way will use up the memory, and the result will be malicious (called slashing), crippling the whole system. The situation has been already discussed in details in an article of another series.

If I suffer to deform the code structure by interweaving the string consumption logic into the string generation logic, the memory space exhaustion will be prevented, but I will not suffer so: the string generation logic should just generate the string while the string consumption logic should just consume the string.

---

2: Java Has 'java.io.PipedWriter' + 'java.io.PipedReader'

Hypothesizer 7
In fact, Java has 'java.io.PipedWriter' + 'java.io.PipedReader', which conveys any string serially, using only a memory space of the pipe size.

The problem is solved, for Java!

---

3: Python Does Not . . .

Hypothesizer 7
I thought that there would be a Python equivalent, . . . but could not find any. Hmm . . .

---

4: The Plan

Hypothesizer 7
So, I have no choice but to create my string pipe.

In fact, I already have an objects pipe, which can be used as a string pipe, because string is a sequence of characters, which are objects. In fact, as Python has no character type (which I deem unwise), the 'str' type will have to represent each character, which is inefficient, but will have to be endured.

However, some additional methods (like one reading a line) will be handy for string pipe, so, I will extend the objects pipe class.

---

5: The Code

Hypothesizer 7
This is the code, which has mypy annotations (as my any Python code does).

theBiasPlanet/coreUtilities/pipes/StringPipe.py

@Python Source Code

from typing import List
from typing import Optional
from typing import TextIO
import sys
from theBiasPlanet.coreUtilities.constantsGroups.GeneralConstantsConstantsGroup import GeneralConstantsConstantsGroup
from theBiasPlanet.coreUtilities.inputsHandling.NoMoreDataException import NoMoreDataException
from theBiasPlanet.coreUtilities.inputsHandling.NoMoreNeedsException import NoMoreNeedsException
from theBiasPlanet.coreUtilities.pipes.ObjectsPipe import ObjectsPipe
from theBiasPlanet.coreUtilities.timersHandling.TimeOutException import TimeOutException

class StringPipe (ObjectsPipe [str]):
	def __init__ (a_this: "StringPipe", a_bufferLength: int, a_notificationIsDelayed: bool) -> None:
		
		super ().__init__ (a_bufferLength, a_notificationIsDelayed)
	
	def __del__ (a_this: "StringPipe") -> None:
		None
	
	def writeWholeString (a_this: "StringPipe", a_reader: TextIO) -> None:
		l_writtenCharacter: Optional [str] = None
		while True:
			l_writtenCharacter = a_reader.read (1)
			if l_writtenCharacter == "":
				break
			try:
				a_this.write (l_writtenCharacter)
			except (NoMoreNeedsException) as l_exception:
				break
	
	def readString (a_this: "StringPipe", a_maximumLength: int, a_timeOutPeriodInMilliseconds: int = -1) -> str:
		l_readCharacter: Optional [str] = None
		l_readStringBuilder: List [str] = []
		l_readStringLength: int = 0
		while l_readStringLength < a_maximumLength:
			try:
				if l_readStringLength == 0:
					l_readCharacter = a_this.read (a_timeOutPeriodInMilliseconds)
				else:
					l_readCharacter = a_this.read ()
			except (NoMoreDataException, TimeOutException) as l_exception:
				if l_readStringLength == 0:
					raise l_exception
				break
			if l_readCharacter is not None:
				l_readStringBuilder.append (l_readCharacter)
				l_readStringLength = l_readStringLength + 1
		return "".join (l_readStringBuilder)
	
	def readStringLine (a_this: "StringPipe", a_maximumLength: int, a_timeOutPeriodInMilliseconds: int = -1) -> str:
		l_readStringBuilder: List [str] = []
		l_readStringLength: int = 0
		while True:
			try:
				l_readCharacter: str = a_this.read (a_timeOutPeriodInMilliseconds)
				if l_readCharacter is not None:
					l_readStringBuilder.append (l_readCharacter)
					l_readStringLength = l_readStringLength + 1
					if l_readCharacter == GeneralConstantsConstantsGroup.c_newLineCharacter or l_readStringLength >= a_maximumLength:
						break
			except (NoMoreDataException, TimeOutException) as l_exception:
				if l_readStringLength == 0:
					raise l_exception
				break
		return "".join (l_readStringBuilder)
	
	def readWholeString (a_this: "StringPipe") -> str:
		l_readCharacter: Optional [str] = None
		l_readStringBuilder: List [str] = []
		while True:
			try:
				l_readCharacter = a_this.read ()
			except (NoMoreDataException) as l_exception:
				break
			if l_readCharacter is not None:
				l_readStringBuilder.append (l_readCharacter)
		return "".join (l_readStringBuilder)

theBiasPlanet/coreUtilities/constantsGroups/GeneralConstantsConstantsGroup.py

@Python Source Code

~

class GeneralConstantsConstantsGroup:
	~
	c_newLineCharacter: str = '\n' # char
	~

I leave a memo here that I have not intensively tested the code, yet, although I have used it in some cases, one of which is shown in the next section.

---

6: An Example Usage and an Execution Result

Hypothesizer 7
This is an example usage.

@Python Source Code

from typing import List
from io import StringIO
import sys
from threading import Thread
from theBiasPlanet.coreUtilities.inputsHandling.NoMoreDataException import NoMoreDataException
from theBiasPlanet.coreUtilities.inputsHandling.NoMoreNeedsException import NoMoreNeedsException
from theBiasPlanet.coreUtilities.pipes.StringPipe import StringPipe

class Test1Test:
	@staticmethod
	def main (a_arguments: List [str]) -> None:
		Test1Test.test3 ()
	
	@staticmethod
	def test3 () -> None:
		l_stringPipe: "StringPipe" = StringPipe (16, True)
		def l_subThreadFunction () -> None:
			try:
				Test1Test.writeString (l_stringPipe)
			except (Exception) as l_exception:
				None
			finally:
				try:
					l_stringPipe.finishWriting ()
				except (Exception) as l_exception:
					None
		l_subThread: Thread = Thread (target = l_subThreadFunction)
		l_subThread.start ()
		Test1Test.readString (l_stringPipe)
		l_subThread.join ()
	
	@staticmethod
	def writeString (a_writer: StringPipe) -> None:
		l_iterationIndex: int = 0
		for l_iterationIndex in range (0, 512, 1):
			l_string: str = str (l_iterationIndex) + "\n"
			try:
				a_writer.writeWholeString (StringIO (l_string));
			except (NoMoreNeedsException) as l_exception:
				break
			sys.stdout.write ("### written: {0:s}\n".format (l_string))
			sys.stdout.flush ()
	
	@staticmethod
	def readString (a_reader: "StringPipe") ->None:
		l_maximumReadStringLength: int = 10
		while True:
			try:
				l_string: str = a_reader.readStringLine (l_maximumReadStringLength)
			except (NoMoreDataException) as l_exception:
				break
			sys.stdout.write ("### read: {0:s}\n".format (l_string))
			sys.stdout.flush ()

if __name__ == "__main__":
	Test1Test.main (sys.argv)

This is an output.

@Output

### written: 0

### written: 1

### read: 0

### read: 1

### read: 2

### written: 2

### written: 3

### written: 4

### written: 5

### written: 6

### written: 7

### written: 8

### written: 9

### read: 3

### read: 4

### read: 5

### read: 6

### read: 7

### read: 8

### read: 9

### written: 10

### written: 11

### written: 12

### written: 13

### written: 14

### written: 15

### read: 10

### read: 11

### read: 12

### read: 13

### read: 14

### read: 15

~
### written: 496

### read: 493

### read: 494

### read: 495

### read: 496

### written: 497

### written: 498

### written: 499

### written: 500

### read: 497

### read: 498

### read: 499

### read: 500

### written: 501

### written: 502

### written: 503

### written: 504

### read: 501

### read: 502

### read: 503

### read: 504

### written: 505

### written: 506

### written: 507

### written: 508

### read: 505

### read: 506

### read: 507

### read: 508

### written: 509

### written: 510

### written: 511

### read: 509

### read: 510

### read: 511

---

References

<The previous article in this series | The table of contents of this series | The next article in this series>